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NAVTRAEQUIPCEN  78-C-0151-1 
FOREWORD 


The  system  described  by  this  report  has  not  been  validated.  This  report 
represents,  instead,  the  efforts  which  led  to  the  implementation  of  a  labora¬ 
tory  system  which  will  allow  testing  of  the  concept  of  a  closed-loop  LSO  train¬ 
ing  system.  Validation  and  refinement  of  the  software  described  in  this  report 
will  provide  data  for  use  in  the  development  of  an  operational  LSO  training 
system. 
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training  considerations . 
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SECTION  I 
INTRODUCTION 

This  report  describes  research  activities  and  results  which  are  concerned 
with  application  of  modern  training  technology  to  a  Navy  LSO  training  problem. 
The  Navy's  LSO  training  program  is  producing  insufficient  numbers  of  skilled 
LSOs.  The  level  of  skill  acquired  in  training  is  also  under  scrutiny.  An 
earlier  study  sponsored  by  the  Naval  Training  Equipment  Center 
(NAVTRAEQUIPCEN)  analyzed  the  LSO  Job  and  training  process1.  That  study 
resulted  in  the  identification  of  LSO  training  program  shortcomings  and  pro¬ 
vided  recommendations  for  improvement,  including  the  description  of  an  auto¬ 
mated  LSO  training  system  concept.  Based  on  recommendations  from  that  study, 
a  program  was  initiated  to  develop  and  utilize  a  laboratory  LSO  training  sys¬ 
tem  for  the  investigation  of  automated  LSO  training  system  concepts. 

The  laboratory  training  system  study  is  the  subject  of  this  report.  It 
was  conducted  in  two  phases.  The  initial  phase  covered  the  time  frame  between 
September  1978  and  September  1979.  The  follow-on  phase  was  performed  between 
October  1979  and  September  1980.  There  have  been  other  related  activities 
sponsored  by  NAVTRAEQUIPCEN  during  this  time  frame.  Subjects  of  these  studies 
include  modelling  LSO  "waving"  behavior,  identification  of  global  measures  of 
LSO  "waving"  effectiveness  and  evaluation  of  the  LSO  Reverse  Display,  an  oper¬ 
ational  device  designed  to  support  LSO  training.  NAVTRAEOUIPCEN  has  also  been 
conducting  in-house  studies  of  LSO  training  strategies  and  applications  of 
computer  aided  LSO  instruction. 

The  results  of  this  program  have  confirmed  the  feasibility  of  the  auto¬ 
mated  LSO  training  .ystem  concept  and  have  provided  positive  indications  of 
potential  training  f fectiveness  for  such  a  concept. 

Subsequent  port  ions  of  this  report  describe  program  objectives,  activi¬ 
ties,  results  and  recommendations  for  future  program  direction.  An  appendix 
is  also  provided  whi ch  describes  the  recommended  performance  capabilities  for 
an  experimental  prototype  LSO  training  system. 


1.  J.T.  Hooks,  E.A.  Butler,  R.A.  Gullen  and  R.J.  Petersen,  Design  Study  for 
an  Auto-Adaptive  LSO  Training  System,  NAVTRAEQUIPCEN  77-C-O 109-1,  December 
1978. 
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SECTION  II 
PROGRAM  OBJECTIVES 

The  purpose  of  this  program,  as  stated  in  the  Study  Specification,  was  to 
"...  empirically  refine  and  validate  the  functional  requirements  and  perform¬ 
ance  specifications  for  an  automated  adaptive  LSO  training  system...."  This 
was  to  be  accomplished  "...  through  the  development,  implementation  and 
utilization  of  a  laboratory  system."  Program  activities  toward  this  purpose 
covered  two  separate  phases,  an  initial  contract  effort  and  a  follow-on 
contract  effort. 

The  objectives  of  the  initial  contract  phase  were  to  make  significant 
progress  in  the  resolution  of  LSO  training  system  design  uncertainty  and  to 
demonstrate  the  automated  LSO  training  system  concepts  which  were  derived  in 
an  earlier  study  by  Hooks  and  others  (1978).  In  support  of  these  objectives, 
several  areas  of  investigation  were  planned.  They  are  delineated  in  Table  1. 
Of  primary  interest  were  the  concepts  of  a  "pilotless"  LSO  training  system, 
automated  LSO  performance  measurement,  and  the  requirements  for  visual  simu¬ 
lation  in  an  LSO  training  system.  These  were  considered  the  most  important 
issues  in  resolving  the  functional  requirements  for  an  automated  adaptive  LSO 
training  system.  System  development  time  was  greater  than  originally  antici¬ 
pated.  This  factor  precluded  comprehensive  experimental  investigations.  Two 
issues,  training  transfer  and  adaptive  LSO  training  strategies,  were  not 
addressed  due  to  limitations  in  the  capabilities  which  could  be  incorporated 
into  the  laboratory  system. 

The  objectives  of  the  second  contract  phase  were  based  on  results  ob¬ 
tained  from  exercising  the  laboratory  LSO  software.  The  first  objective  was 
to  enhance  this  system  from  two  perspectives:  perceptual  characteristics  of 
the  display  and  instructional  features.  The  second  objective  was  to  evaluate 
the  merits  of  interactive  decision-oriented  instruction  early  in  the  LSO 
training  cycle.  A  third  objective  was  to  evaluate  the  feasibility  of  limited 
instructorless  LSO  training.  The  final  objective  was  to  evaluate  the  utility 
of  a  simplified  LSO  decision-making  model  for  early  LSO  training.  There  was 
also  an  underlying  secondary  objective  in  this  contract  phase,  the  evaluation 
of  the  system  itself  as  a  candidate  part-task,  LSO  decision  training  device. 
Some  success  was  obtained  in  meeting  these  objectives,  but  the  r  suits  were 
limited  from  a  scientific  basis.  Although  the  training  effectiveness  of  the 
laboratory  system  was  not  validated,  some  insight  was  gained  into  directions 
for  future  research  with  automated  LSO  training  system  concepts. 
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TABLE  1.  AREAS  OF  INVESTIGATION 
System  Level:  Training  Transfer 


Visual  En\ ironment: 

Pi  lot/ Aircraft  Modellii  g: 
Instructional  Feedback 
Selection  of  Learning  Mternatives : 
Performance  Measurement : 


"Pilotless"  Training  System  Feasibility 

Instructional  Feedback  Display 
Artificial  Cueing 
Resolution  Requirements 
Aircraft  Level  c  f  Detail  Requirements 
Field  of  View  Rf  quirements 

Pilot  Skill  Var  ation 

Relevant  Feedback  Content 

Adaptive  LSO  Training  Strategies 

Speech  Recognition  Techniques 
Applicability  of  LSO  Behavioral  Models 
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PROGRAM  ACTIVITIES 

The  primary  activities  during  this  program  involved  the  development  and 
utilization  of  a  laboratory  LSO  training  system  called  Interactive  Experi¬ 
ments  System  (IES).  Another  activity  invo’ved  the  experimental  utilization  of 
the  Aviation  Wide  Angle  Visual  System  (AWAVS),  now  called  the  Visual 
Technology  Research  System  (VTRS). 

LABORATORY  LSO  SOFTWARE  (IES) 

Development  or  software  for  a  laboratory  LSO  training  system  was  initial¬ 
ly  oriented  to  the  demonstration  of  an  automated  LSO  training  system  concept. 
This  initial  contract  phase  occurred  between  September  1978  and  September 
1979.  Following  that  effort,  a  second  contract  phase  was  initiated  to  inves¬ 
tigate  more  specific  aspects  within  the  LSO  training  system  concept.  This 
activity  occurred  between  October  1979  and  September  1980.  The  activities 
involved  in  these  two  phases  are  described  below. 

INITIAL  PHASE .  As  mentioned  earlier,  this  initial  contract  phase  was  oriented 
toward  several  areas  of  investigation.  For  IES,  the  primary  rocus  was  on 
development  and  demonstration  of  the  concept  of  an  automated  system  for  LSO 
training:  a  "pilotless,"  closed- loop  system  allowing  LSO  interaction  through 

automated  speech  recognition.  Figure  1  depicts  the  equipment  configuration  of 
IES. 

Development.  Development  of  this  system  was  based  on  a  preliminary  design 
presented  in  an  earlier  report  by  Hooks  and  others  (1978).  Hardware  at 
NAVTRAEQUIPCEN  for  which  IES  was  designed  included  a  NOVA  1200  CPU,  a  NOVA  800 
CPU  with  floating  point,  a  VIP-100  voice  recognition  preprocessor,  an  inter- 
orocessor  communication  link  (IPB),  a  shared  disk  featuring  two  removable  and 
two  fixed  cartridges,  a  Megatek  5000  random  scan  graphics  display,  and  a 
Talley  Model  2200  line  printer.  Software  design  and  development  were  accom¬ 
plished  by  Logicon  and  NAVTRAEQUIPCEN  personnel,  using  a  government-owned 
Eclipse  S-130  at  Logicon.  IES  was  designed  such  that  Logicon  personnel  pro¬ 
duced  the  software  for  system  elements  to  be  run  on  the  NOVA  800  and 
NAVTRAEQUIPCEN  personnel  developed  the  software  elements  for  the  NOVA  "i  2 0 0  . 
This  minimized  system  zesting  and  integration  problems  during  development . 
Early  into  the  development  effort  it  was  noted  that  there  would  be  limitations 
in  CPU  capacity  which  would  limit  the  number  of  available  training  functions. 
Therefore  development  efforts  had  to  include  functional  priority  considera¬ 
tions.  As  a  result,  IES  capabilities  were  somewhat  less  than  originally 
envisioned,  and  the  operating  system  for  the  Nova  d00  was  changed  from  ROC’S,  a 
disk  based  system,  to  RTCS,  a  memory  resident  system;  RTOS  has  fewer  capabili¬ 
ties  but  is  also  Less  demanding  of  resources. 

Software  pefr.-r.  :  :>i.  ;.on.  ’.he  opera  ..im,  concept  i  or  was  to  a!'  .  »-  trance  to 

view  a  graphics  portrayal  of  an  approach,  perceive  the  need  Lor  a  --oi.ee  call 
and  it;  v.v  the  call  to  tin  system.  The  system  would  then  process  t.,e  call, 
sirr.ul  ite  and  display  .lot  rest  -  sc,  and  recced  an  evaluation  or  trainer  per¬ 
form.-,  re.  Thus.  iEC  o  videe  sec- loop  LS  and  pilot  interact  Jen  m  an 
l nstructional  set nan o -control led  environment . 
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There  are  several  major  functional  aspects  to  IES:  graphic  display  of 
aircraft  and  approach  scene,  LSO  performance  evaluation,  speech  recognition, 
Aircraft/Pilot  Model  (APM),  exercise  control  and  performance  data  recording. 
Paragraphs  below  provide  descriptions  of  system  functions  and  an  overview  of 
system  operations  in  its  initial  version. 

The  graphic  system  of  IES  provides  a  line-drawing  depiction  of  an 
approaching  aircraft  and  background  scene  including  horizon,  carrier  deck  out¬ 
line  and  ship's  wake.  It  also  provides  text  messages  and  an  aircraft  position 
grid  presenting  dynamic  glideslope,  lineup  and  range  information.  The  air¬ 
craft  image  has  minimal  fidelity,  looking  somewhat  like  a  wire  model  airplane. 
This  significantly  minimizes  graphic  processing  requirements.  The  visual 
field  of  view  covers  about  60  degrees  (horizontal)  and  the  aircraft  image  is 
updated  approximately  12  times  per  second.  Figure  2  is  an  annotated  depiction 
of  IES  in  the  initial  phase. 

LSO  performance  evaluation  is  provided  by  correlating  recognized  voice 
call  input  to  aircraft  state  parameters  (including  glideslope,  lineup,  angle 
of  attack  and  range)  to  a  pre-programmed  model  of  correct  performance.  Table 
2  provides  a  listing  of  33  voice  call  and  approach  situation  correlations 
which  are  incorporated  into  IES.  This  is  essentially  a  significant  sim¬ 
plification  of  the  LSO  behavioral  model  developed  by  Borden  and  McCauley 
(1978). 2  Performance  evaluation  in  IES  is  a  "snapshot"  model  which  does  not 
account  for  rates  of  aircraft  parameter  changes  (such  as  sink  and  drift  rate). 
However,  the  model  implementation  is  such  that  it  can  be  increased  in  complex¬ 
ity  if  IES  were  implemented  on  hardware  with  greater  CPU  resources.  Another 
limitation  to  the  initial  version  of  performance  evaluation  is  that  "errors  of 
omission"  (e.g.  no  voice  call  when  one  is  required)  are  not  detected. 

Automated  speech  recognition  involves  the  collection  of  an  LSO's  voice 
patterns  on  each  call  which  will  later  be  used  in  training  exercises.  This  is 
the  interactive  link  between  the  LSO  and  the  approach  situation.  IES  has  been 
designed  to  handle  23  standard  LSO  voice  calls.  Table  3  provides  a  listing  of 
IES  voice  calls.  Not  all  of  them  are  used  in  the  performance  evaluation 
function . 

The  aircraft/pi lot  model  (APM)  function  guides  the  simulated  aircraft 
flight  dynamics  based  on  pre-programmed  maneuver  commands  or  commands  based  on 
LSO  voice  calls.  The  approach  speed  of  the  aircraft  is  fixed  at  110  knots  of 
closure.  Glideslope  and  lineup  positioning  vary  between  a  series  of  zones 
which  reflect  various  deviations  from  optimum  positioning  during  approach. 
Rates  of  glideslope  and  lineup  positioning  changes  are  variable  pre-programmed 
values.  Lineup  positioning  changes  include  aircraft  roll  movement  for  change 
initiation  and  completion.  Angle  of  attack  (AOA)  is  reflected  in  pitch  vari¬ 
ation  between  optimum  (about  10  degrees  nose  up),  fast  (about  five  decrees 
nose  up)  and  slow  (about  fifteen  degrees  nose  up).  Range  is  segmented  into 
four  zones  wnich  begin  at  one  mile:  "start,"  "in  the  middle,"  "i,  .  ..  "  and 

"at  the  ramp."  Figure  3  provides  unsealed  depictions  of  gl  '.desiope ,  a  up 

and  rancre  zone  variations.  APM  also  reflects  pilot  skill  and  responsiveness 


2.  G.J.  Borden  and  M.E.  McCauley,  Computer  Based  I, SO  Carrier  Aircraft 
recovery  Model  (Progress  Report),  Human  Performance  Research,  Inc.,  1978. 
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TABLE  2.  PERFORMANCE  MODEL  IN  IES 


Relevant  Aircraft  Parameters 


Correct  Call 


"You1 

re 

High" 

"You' 

re 

High" 

"You1 

re 

High" 

"You' 

’  re 

Low" 

"You1 

re 

Low" 

"You1 

1  re 

Lined 

Up 

Left" 

"You1 

1  re 

Lined 

Up  Left" 

"You 1 

're 

Lined  Up 

Right 

"You1 

1  re 

Lined 

Up 

Right' 

"You1 

1  re 

Fast" 

"You' 

1  re 

Fast" 

"You1 

re 

Slow" 

"You1 

1  re 

Slow” 

"Power" 

"Power" 

"Right  for  Line  Up" 
"Left  for  Line  Up" 
"Waveof f" 

"Waveoff " 

"Waveof f" 

"Waveoff" 

"Waveoff" 

"Waveoff" 

"Waveof  f" 

"Waveoff" 

"Waveoff" 

"Waveoff" 

"Waveof  f" 

"Waveoff" 

"Waveoff" 

"Waveoff" 

"Waveoff" 

"Waveoff" 


Range 

Glid. 

aslope 

Start 

Very 

High 

Middle 

Very 

High 

Close 

Very 

High 

Start 

Low 

Middle 

Low 

Start 

— 

— 

Middle 

— 

Start 

— 

— 

Middle 

— 

— 

Start 

— 

Middle 

— 

Start 

— 

Middle 

— 

Close 

Low 

Close 

— 

Close 

— 

Close 

— 

Close 

Low 

Close 

Low 

Close 

High 

Close 

High 

Close 

Low 

Close 

High 

Close 

— 

— 

Close 

— 

Close 

— 

Close 

— 

Ramp 

Low 

Ramp 

High 

Ramp 

— 

Ramp 

— 

Ramp 

— 

Ramp 

— 

Line  Up  AOA 


Left  - 

Left  - 

Right  - 

Right  - 

-  Fast 

-  Fast 

-  Slow 

-  Slow 


Slow 


Left 

— 

Right 

— 

Left 

— 

Right 

— 

Left 

---- 

Right 

— 

Slow 

Fast 

Left 

Slow 

Left 

Fast 

Right 

Slow 

Right 

Fast 

Left 

_ 

Right 

— 

Fast 

Slow 


Note:  " - "  indicates  that  this  parameter  is  ignored  by  the  system. 


14 


NAVT RAEQU I  PC  EN  78-C-0 151-1 
TABLE  3.  LSO  VOICE  CALLS  IN  IES 
Roger  Ball 

You're  A  Little  High 

You're  High 

You're  A  Little  Low 

You're  Low 

You're  Going  High 

You're  Going  Low 

You're  Lined  Up  Left 

You're  Lined  Up  Right 

You're  Drifting  Left 

You're  Drifting  Right 

You're  Fast 

You're  Slow 

Check  Your  Line  Up 

Don't  Settle 

Don't  Go  Low 

Don't  Climb 

Don't  Go  High 

A  Little  Power 

Power 

Right  For  Line  Up 
Left  For  Line  Up 
Waveoff 
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RIGHT 

LITTLE  RIGHT 

ON  LINE  UP 

LITTLE  LEFT 
LEFT 


LINEUP 


START 


Figure  3.  Glideslope,  Lineup  and  Range  Zones 
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variations  based  on  pre-programmed  scenario  inputs*  Skill  is  reflected  in  the 
size  of  glideslope  and  lineup  variation  within  a  zone*  Responsiveness  is 
reflected  by  pre-programmed  time  variations  between  LSO  voice  call  and 
aircraft  response 

A  primary  influence  on  exercise  control  is  the  scene rio  selected  by  the 
operator.  The  scenario  file  contains  a  pr e-programmed  a  rcraft  approach  pro¬ 
file  which  has  been  designed  for  a  specific  training  purpose.  Maneuvers  and 
changes  in  AOA  are  keyed  in  the  scenario  to  range  values.  In  the  initial 
version  of  IES,  text  prompting  messages  are  also  pre-programmed,  and  display 
of  the  text  is  an  operator  option  prior  to  starting  an  approach.  Another  in¬ 
fluence  on  exercise  control  occurs  when  an  LSO  voice  call  is  recognized  by 
IES.  For  a  correct  voice  call  (as  determined  by  performance  evaluation),  the 
aircraft  responds  with  a  pre-programmed  maneuver  to  correct  the  existing 
deviation.  Incorrect  voice  calls  are  ignored  in  the  initial  version  of  IES. 
The  final  influences  on  exercise  control  are  operator  options  selected  prior 
to  commencement  of  an  approach.  In  the  phase  one  version,  these  incLude 
display  of  scenario  text  messages,  display  of  performance  feedback  messages, 
display  of  aircraft  position  grid  and  determination  of  whether  the  aircraft 
will  maneuver  in  response  to  LSO  voice  calls. 

Performance  data  from  an  approach  is  saved  and  displayed  at  the  oper¬ 
ator's  terminal  at  the  end  of  approach  and  is  available  for  printout  at  the 
end  of  a  training  session.  Data  available  in  the  initial  IES  version  includes 
scenario,  LSO  name,  operator  options  selected,  voice  calls  along  with  the 
aircraft  parameters  at  the  time  of  the  call,  correct  call  if  call  made  was 
incorrect,  aircraft  parameters  throughout  the  approach  (at  a  sampling  rate  of 
about  once  every  second),  and  an  accounting  of  time  spent  by  the  aircraft  in 
the  various  glideslope,  lineup  and  AOA  zones. 

From  an  overall  viewpoint,  IES  is  a  limited  representation  of  all  major 
functional  elements  of  an  automated  LSO  training  system  with  the  exception  of 
automated,  adaptive  syllabus  control. 

Utilization  and  Results.  After  implementation  the  system  was  demonstrated  to 
several  fleet  LSOs,  and  an  informal  test  was  conducted  in-house  at  Logicon. 

Two  subjects  were  used  in  the  test,  each  having  a  Navy  carrier  pilot  back¬ 
ground,  but  no  LSO  experience.  Over  two  days,  each  subject  was  sequenced 
through  syllabus  exercises.  Syllabus  sequencing  was  based  on  incremental 
introduction  of  waving  skill  components  (glideslope,  lineup,  AOA  and  waveoff 
related  calls) .  From  the  standpoint  of  system  operability,  the  system  tested 
successfully.  From  a  training  standpoint,  however,  several  deficiencies  were 
discovered. 

a.  One  of  the  major  discrepancies  involved  difficulti< s  on  the  percep¬ 
tion  of  approach  deviations  presented  by  the  scenarios.  This  led  to  a  con¬ 
clusion  that  some  amount  of  parameter  exaggeration  was  required  to  enable 
meaningful  presentation  of  waving  situations. 

b.  Absence  of  a  capability  to  detect  errors  of  omissions  sir  nificantly 
lessened  the  value  of  the  performance  evaluation  function. 

c.  It  became  evident  during  testing  that  a  freeze  capability  would  aid 
effective  conduct  of  a  training  session. 
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d.  The  aircraft  position  grid  proved  to  be  excessively  cluttered,  thus 
lessening  its  effectiveness  as  a  perception  aid. 

e.  The  fact  that  the  aircraft  responded  only  to  correct  LSO  calls  turned 
out  to  be  an  excessive  departure  from  real-world  LSO  waving  interaction. 

f.  There  were  speech  recognition  difficulties  which  were  attributed  to 
lack  of  a  contextual  voice  data  collection  (VDC)  and  to  collection  of  data  for 
more  calls  than  were  actually  used.  The  VDC  capability  was  implemented  at 
NAVTRAEQUIPCEN  but  was  not  used  in  Logicon's  in-house  testing. 

g.  It  was  noted  that  the  performance  evaluation  feedback  feature  would 
have  been  more  effective  if  clarifying  information  could  have  accompanied  the 
display  of  the  correct  call. 

h.  Design  of  the  glideslope  and  lineup  "zones"  to  converge  at  the  opti¬ 
mum  touchdown  position  significantly  degraded  the  depiction  of  deviations  for 
the  "in  close"  and  "at  the  ramp"  segments  of  the  approach,  precluding  vari¬ 
ations  in  the  final  results  of  the  approach.  The  system  was  thus  unable  to 
provide  additional  feedback  on  the  effectiveness  of  LSO  control  in  the 
terminal  portion  of  the  approach. 

i.  There  was  some  question  regarding  the  fidelity  of  the  aircraft  image 
from  a  training  standpoint. 

j.  It  was  also  noted  that  some  form  of  evaluative  feedback  should  be 
presented  to  the  subject  by  the  system  following  the  approach,  in  order  to 
enhance  learning  rate. 

Another  result  of  testing  and  demonstration  of  the  system  to  skilled  LSOs 
was  the  realization  that  the  laboratory  system  with  refinement,  has  signifi¬ 
cant  potential  as  an  introductory,  part-task  LSO  decision  training  system.  It 
could  provide  limited  interactive  training  in  an  instructional  setting  such  as 
the  LSO  Phase  I  School. 

SECOND  PHASE.  In  response  to  the  results  obtained  from  initial  utilization  of 
XES ,  software  revision  and  IES  testing,  in  accordance  with  the  program  objec¬ 
tives  stated  earlier,  were  conducted  next. 

Development.  Since  the  software  revisions  were  directed  at  increasing  the 
capabilities  of  IES,  the  initial  efforts  involved  identification  of  candidate 
enhancements  and  the  structuring  of  overlays  in  IES  to  increase  available 
memory.  A  functional  specification  of  highest  priority  enhancements  was 
developed  and  the  software  revisions  were  designed  and  implemented  incre¬ 
mentally.  As  in  the  initial  contract  pnase,  the  revision  effort  involved 
Logicon  and  NAVTRAEQL'IPCEN  programmers.  The  functional  aspects  of  the  system 
which  were  revised  included  a  graphic  system,  performance  evaluation,  APM, 
exercise  control  and  performance  data  recording.  The  scenario  files  were  also 
nodified  and  additional  scenarios  were  added  to  IES.  The  results  of  the 
revisions  follow. 


Software  Description.  The  operating  concept  of  IES  remains  as  described  for 
•-he  initial  contract  phase.  The  revisions  for  this  contract  phase  are  out¬ 
lined  in  Table  4  and  discussed  in  the  paragraphs  below. 
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Freeze  was  implemented  in  the  revised  IES.  This  is  initiated  through  the 
operator  terminal.  From  the  "freeze”  state,  operator  action  can  be  taken  t  > 
either  resume  the  approach  to  its  completion  or  terminate  the  approach. 

TABLE  4.  IES  ENHANCEMENTS 


Freeze 

Prompt  and  Feedback  Messages 

Detection  of  Voice  Call  Omissions  by  Performance  Evaluation 
System 

Graphic  System  Enhancements 

Aircraft  Response  to  Selected  Incorrect  Calls 
Revision  of  Glideslope  and  Lineup  Zone  Origin 
Exaggeration  of  Deviations 
Approach  Results  Display 
Revision  of  Performance  Summary  Data 

During  freeze,  text  messages  and  aircraft  position  grid  remain  displayec . 
During  freeze,  an  enhanced  aircraft  image  is  displayed. 

Prompt  and  feedback  text  messages  were  expande  1  to  provide  clarifying 
information  in  terms  of  relevant  aircraft  approach  parameters.  For  exar.ple, 
if  the  call  "You're  Low"  was  recognized,  but  "You'r;  High"  was  the  corr.  ct 
call,  the  relevant  parameters  of  range  and  glideslcse  would  be  displayed.  The 
display  of  prompt  messages  was  revised  such  that  they  are  now  data-driven  and 
rot  controlled  by  pre-programmed  scenario  t  ixt  commands.  If  the  prompt  option 
is  selected,  the  correct  call  and  relevant  parameters  are  displayed  as  soon  as 
the  aircraft  reaches  a  state  requiri  ig  a  call. 

Performance  evaluation  accounts  for  "errors  of  omission"  in  the  revised 
IES.  If  a  call  is  not  made  within  a  brief  time  period  after  the  aircraft 
reaches  a  state  requiring  a  call,  an  LSO  performance  error  is  noted  by  the 
system.  If  the  feedback  message  option  is  selected,  information  as  described 
above  for  a  prompt  is  displayed. 

There  were  several  revisions  to  the  IES  graphic  system.  One  mentioned 
earlier  is  the  enhanced  aircraft  image  displayed  during  freeze.  This  image  is 
similar  co  t hi  A- 7  aircraft  and  provi  ies  perceivable  separatio  of  wing  and 
horizontal  stc-n.izer.  The  ri.n-time  aircraft  image  was  also  improved  slightly. 
Additional  carrier  landing  area  markings  were  incorporated  into  the  background 
scene.  Clutter  ir.  the  aircraft  position  grid  was  reduced.  Figure  4  is  a 
depiction  of  the  revised  IES. 

The  capability  was  added  for  the  aircraf-  to  respond  to  selected  incor¬ 
rect  voice  calls.  These  include  "waveoff,"  “power,"  and  ''right/left  for 
lineup”,  which  are  imperative  LSO  calls  for  which  pilot  response  is  very 
likely  even  if  incorrectly  used  by  the  LSO. 

The  origin  for  glideslope  and  lineup  zones  was  revised.  The  initial  ver¬ 
sion  of  IES  had  the  zone  originating  from  the  ideal  touchdown  point  on  the 
deck.  This  did  not  allow  foi  deviations  on  touchdown,  nor  did  it  provide  ade¬ 
quate  perceptual  variation  ir  the  final  portion  of  the  approach.  The  zone 
orig.n  has  been  roved  so  that  glideslope  and  lineup  deviations  on  touchdown 
can  vary  a  ,d  be  discriminated  by  the  system  for  reporting  approach  results  to 
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the  LSO.  In  response  to  difficulties  reported  in  perceiving  glideslope, 
lineup  and  AOA  deviations,  the  sizes  of  the  deviations  were  exaggerated  and 
optimum  glideslope  was  rotated  upward. 

At  the  end  of  an  approach,  touchdown  dynamic-  are  presented  in  the  text 
area  of  the  display.  Parameters  presented  include  arresting  wire  (where  a  3 
wire  is  optimum),  lineup  position  (only  if  other  than  optimum)  and  angle  of 
attack  (only  if  slow  or  fast).  If  the  aircraft  is  too  high  to  catch  a  wire 
the  landing  result  presented  is  "bolter"  (landed  but  did  not  catch  a  wire)  or 
"waveoff"  (too  high  to  touch  the  deck). 

The  system  fo.-  performance  summary  data  recording,  displaying  it  (on  the 
operator  terminal)  and  making  it  available  for  print-out  was  revised.  The 
periodic  recording  of  aircraft  parameters  throughout  the  approach  was  deleted. 
The  summary  of  LSO  voice  calls  correlated  to  aircraft  parameters  was  stream¬ 
lined  and  revised  to  present  word  and  acronym  descriptions  instead  of  number 
codes.  The  zone  accounting  data  was  also  reduced.  Figure  5  shows  a  sample 
performance  summary  print-out. 

As  mentioned  earlier,  IES  is  a  laboratory  representation  of  automated  LSO 
training  system  concepts  reported  earlier  by  Hooks  and  others  (1978).  Figure 
6  excerpted  from  the  report,  presents  a  generic  functional  architecture  for  an 
automated  LSO  training  system.  Table  5  presents  a  summary  of  IES  features 
correlated  to  that  functional  architecture. 

Utilization.  After  Implementation  of  software  revisions,  IES  was  informally 
exercised  with  three  subjects  at  Logicon's  San  Diego  facility.  Time  and 
equipment  availability  constraints  precluded  extensive  experimentation.  The 
intent  of  this  activity  was  to  obtain  performance  data  and  subject  ve  commen¬ 
tary  which  would  help  identify  potential  strengths  and  limitations  of  IES  and 
the  LSO  training  system  concepts  represented  by  it.  Each  subject  eceived 
training  in  the  basic  LSO  decision  skill  subset  which  was  designed  into  IES. 
Questionnaire  and  system  performance  evaluation  data  was  recorded  during  the 
study  to  evaluate  system  capabilities  and  potential  training  merits. 

The  training  consisted  of  four  sessions  for  each  subject.  Each  session 
was  approximately  one  and  one-half  hours  in  duration.  Two  of  the  subjects 
were  Navy  pilots  without  LSO  skills;  one  was  a  very  experiencec  carrier  pilot, 
the  other  very  inexperienced.  The  third  subject  was  a  highly  experienced  LSO. 

There  were  several  procedures  in  the  study  which  were  common  to  each  ses¬ 
sion.  Each  session  included  four  portions:  session  briefing,  training,  test¬ 
ing,  questionnaire  completion.  During  the  session,  the  subject  was  allowed  to 
retain  a  handout  containing  system  and  training  information  for  easy  refer¬ 
ence.  Midway  through  th j  training  portion  of  the  session  and  just  prior  to 
testing,  the  subject  was  given  a  break  from  session  activities.  During  test¬ 
ing,  the  system  operator  manually  recorded  voice  calls  when  speech  recognition 
errors  occurred. 

The  testing  portion  of  each  session  involved  having  the  subject  "wave"  20 
scenarios  without  the  aircraft  position  grid  and  with  only  speech  recognit  .on 
feedback  messages  available.  The  system  operator  provided  no  verbal  feedback 
to  the  subiect  regarding  performance  during  testing.  The  test  always  included 
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Figure  5.  Performance  Data  Printout 
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TABLE  5.  SUMMARY  OF  IES  FEATURES 


INSTRUCTIONAL  PRESENTATION 


a.  Visual  Environment 

1.  generic  aircraft  outline  provided  during  approach;  at  freeze  the 
outline  switches  to  one  with  similarity  to  the  A?  aircraft 

2.  day  scene  operating  environment  with  depiction  of  ship's  wake; 
field  of  view  of  about  60  degrees 

3.  generic  carrier  deck  outline  with  centerline  and  "ladder  line" 
markings;  flush  deck  LSO  platform 

4.  instructional  effects  include  cueing  aid  for  aircraft  positioning 
(glideslope,  lineup,  range)  and  text  messages  for  feedback  and 
prompting 

b.  Pi lot/ Air craft 

1.  simplified  performance  characteristics  included  fixed  approach 
speed,  limited  variations  in  AOA  (fast,  on,  slow),  limited 
variations  in  sink  and  drift  rate,  limited  correlation  of  roll  to 
lineup  changes,  no  correlation  of  pitch  to  glideslope  changes 

2.  "zones"  established  for  glideslope,  lineup  and  AOA  deviations  for 
control  of  variations  in  aircraft  maneuvers 

3.  pilot  characteristics  limited  to  size  of  deviations  within  zones 
I  small,  medium,  large)  and  variations  in  response  rate  for 
maneuvers  (fast,  medium,  slow) 

c.  Audio  Cues  -  not  provided 

d.  Workstation  Displays  -  not  explicitly  provided,  however  cueing  aid 

has  functional  similarity  to  LSO  HUD 

e .  Workstation  Controls  -  microphone  provided  for  LSO  communications  to 

the  pi lorn 

f .  Deck  Mot ior  Cues  -  not  provided 

g.  Instructional  Feedback  -  through  operator  options,  performance  feed¬ 
back  and  prompting  are  available 


ADAPTIVE  CONTROL 


a.  Selection  of  Learning  Alternatives  -  automated  capability  not 
provided 

b.  Selection  of  Proficiency  Alternative  -  automated  capability  not 
provided 
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TABLE  5.  SUMMARY  OF  IES  FEATURES  (CONT’D) 
c.  Instructor  Intervention 


1.  instructor  (operator)  displays  limited;  over- the- shoulder  t bser- 
vation  of  student  display,  own  CRT  display  of  options  selec  ted 
and  end  of  run  summary  data  (approach  results  and  student 
performance) 

2.  instructor  (operator)  control  limited;  scenario  selection,  pre¬ 
selection  of  instructional  options  (prompting,  feedback,  cueing 
aid,  aircraft  response) ,  freeze  during  run,  re-run  approach 

3.  data  automatically  recorded  for  each  approach  (operator  selec¬ 
tions,  student  performance,  approach  results)  and  available  in 
hard  copy  through  a  line  printer 

PERFORMANCE  EVALUATION 

a.  Performance  Measurement 

1.  speech  recognition  for  limited  LSO  Cells 

2.  detection  of  aircraft  state  parameters 

3.  accountability  of  aircraft  positioning  in  "zones" 

b.  Scoring  -  determination  of  correctness  of  LSO  call  (or  lack  of  call) 

with  respect  to  aircraft  state  parameters 
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me  same  20  scenarios.  However,  they  were  presented  in  a  different  sequence 
each  time.  Ten  of  the  scenarios  were  selected  from  the  21  basic  training 
scenarios  used  in  the  training  portion  of  the  study.  The  other  ten  involved 
more  complex,  multi-dimension  deviations  modelled  after  typical  unsuccessful 
approach  profiles  (those  which  lead  to  bolters,  hard  landings  and  ramp 
strikes).  The  test  did  not  include  any  "catch  trials."  Descriptions  of  the 
training  and  testing  scenarios  are  presented  in  Appendix  A. 

The  questionnaire  completed  by  the  subjects  consisted  of  the  same  ques¬ 
tions  for  each  period.  Of  primary  interest  were  problems  experienced  with 
IES,  suggested  IES  improvements,  ease  of  detecting  deviations  and  learning 
achieved  during  the  session.  At  the  end  of  the  final  session,  additional 
questions  regarding  the  subject's  perceptions  of  overall  learning 
achievements,  IES  instructional  strengths  and  potential  IES  utilization 
concepts  were  presented.  A  copy  of  the  questionnaire  is  presented  in  Appendix 
3.  Questionnaire  results  are  discussed  later  in  this  section. 

All  subjects  received  the  same  testing  and  all  completed  the  question¬ 
naire.  The  two  non-LSO  subjects  received  the  same  briefing  and  training  por¬ 
tion  of  each  session.  The  experienced  LSO  subject  observed  during  each 
session  of  another  subject.  Following  this  observation  effort,  he  received  a 
brief  practice  period  with  the  21  basic  training  scenarios  prior  to  being 
tested.  The  four  sessions  experienced  by  the  non-LSO  subjects  are  described 
below . 

In  the  first  session  the  subject  was  given  a  comprehensive  briefing  con¬ 
cerning  the  study,  IES  and  the  training  to  be  accomplished.  A  system  and 
training  information  handout  was  also  provided  to  him  for  reference.  Follow¬ 
ing  this  the  subject  observed  several  approaches  on  IES  for  f amiliarization. 
The  next  step  was  voice  data  collection  (VDC)  for  the  10  LSO  calls  to  be  used 
in  training.  VDC  was  accomplished  through  visual  prompting  from  an  alpha¬ 
numeric  CUT  terminal,  with  four  repetitions  of  each  phrase  and  voice  valida¬ 
tion.  The  training  portion  of  this  session  consisted  of  IES  prompted  pre¬ 
sentation  of  all  21  basic  training  scenarios.  The  aircraft  position  grid  was 
available  for  each  approach  and  the  system  operator  provided  verbal  pre¬ 
prompting  prior  to  each  approach.  Testing  and  questionnaire  completion 
finished  the  session. 

For  the  second  session ,  the  subject  was  given  a  refresher  briefing  on  the 
system  and  the  study.  The  specific  scenarios  to  be  emphasized  in  this  session 
were  also  discussed.  During  this  session,  training  focused  on  10  scenarios 
which  consisted  of  informative  voice  calls  for  glideslope,  lineup  and  AOA  de¬ 
viation  within  "at  the  start"  and  "in  the  middle"  range  zones.  Initially, 
each  of  the  10  scenarios  was  presented  with  verbal  pre-prompting,  real-time 
I2E  prompting  and  the  aircraft  position  grid  displayed.  The  second  time 
through  the  same  scenarios,  there  was  no  prompting,  but  IES  feedback  and  the 
aircraft  position  grid  were  available.  The  final  portion  of  training  involved 
practice  with  the  10  scenarios  plus  presentation  of  several  "eaten  tr_aj." 
scenarios  (those  with  slight  deviations  but  not  requiring  voice  calls). 

During  practice,  the  only  aids  available  were  IES  performance  feedback 
messages.  Testing  and  questionnaire  completion  finished  the  session. 

In  the  third  session  the  subject  was  again  given  a  refresher  briefing  and 
specific  scenarios  to  be  emphasized  were  discussed.  During  the  session. 
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training  focused  on  seven  scenarios  covering  the  imperative  voice  calls  used 
w  th  single  lineup  and  glideslope  deviations  for  "in  close"  and  "at  the  ramp" 
range  zones  The  initial  portions  of  scenario  presentation  were  as  described 
in  the  second  session.  During  the  practici  portion,  training  scenarios 
learned  in  the  previous  session,  as  well  as  "catch  trial"  scenarios,  were  also 
included.  Testing  and  questionnaire  completion  finished  the  session. 

In  the  fourth  session  the  subject  was  again  given  a  refresher  briefing 
and  specific  scenarios  to  be  emphasized  were  discussed.  Training  for  this 
session  focused  on  four  scenarios  which  covered  the  "waveoff"  call  for  mul¬ 
tiple  dimension  deviation  within  the  "in  close"  range  zone.  The  practice 
portion  of  the  session  included  nearly  all  of  the  21  basic  training  scenarios 
plus  a  few  "catch  trials."  Testing  and  questionnaire  completion  finished  the 
session. 

Several  problems  /e re  encountered  during  utilization  of  the  system. 
Approximately  twenty-f  ve  percent  of  the  LSO  voice  calls  were  not  properly 
re. ognized  by  the  speech  recognition  sub-system.  This,  in  turn,  caused  frus¬ 
tration  on  the  part  of  the  subjects  and  required  manual  recording  of  unrecog¬ 
nized  voice  calls  by  the  system  operator.  Unfortunately,  the  collection  of 
voice  data  in  a  training  context  was  not  available  when  experimentation  began. 
This  is  noteworthy  since  experience  in  other  speech  recognition-based  training 
systems  suggests  that  the  voice  data  collection  procedure  influences  the 
speech  recognition  performance.  The  interested  reader  is  referred  to  Breaux 
and  Goldstein  (1975)  and  Breaux  and  Grady  (1976).  Also,  in  a  follow-up  inves¬ 
tigation  of  the  speech  recognition  subsystem,  it  was  found  that  the  threshold 
parameters  were  not  optimized  for  naive  system  users.  Another  problem  was 
that  there  was  no  performance  data  recorded  for  one  of  the  subjects  (inexperi¬ 
enced  pilot)  during  his  first  two  sessions.  This  was  due  to  a  temporary 
breakdown  in  the  data  recording  feature  of  IES.  There  was  ai  excessive  delay 
of  aircraft  response  to  scenario-generated  maneuver  commands  and  to  LSO  voice 
calls.  This  discrepancy  was  noted  prior  to  experimentation,  and  scenario 
maneuver  timing  was  corr acted  to  better  reflect  scenario  training  objectives. 
However,  delays  in  aircraft  response  to  LSO  calls  was  not  corrected.  Prom  a 
"waving"  standpoint,  the  aircraft  response  delays  lessened  the  realism  of 
LSO-pilot  interaction  an  1  was  noticeable  to  the  subjects.  Another  problem  was 
that  the  output  from  performance  evaluation  was  occasionally  in  error.  For 
example,  the  recorded  output  of  aircraft  parameters  did  not  always  agree  with 
the  voice  call  which  was  recorded  as  correct  or  incorrect.  This  error  was 
very  infrequent  but  required  extra  attention  to  the  performance  data  printouts 
during  analysis. 

Several  other  items  concerning  experimentation  are  considered  wc rthy  cf 
comment.  A  few  features  of  IES  were  not  exercised  during  experiment,  tion. 

The  operator  options  to  freeze,  to  rerun  scenario,  end  to  preclude  aircraft 
maneuvers  on  LSO  calls  were  not  used.  The  training  value  of  freeze  and  rerun 
were  considered  unquestionable  and  therefore  were  intentionally  left  out  since 
this  was  such  a  brief  study.  Precluding  aircraft  maneuvers  on  LSO  voice  c,  11s 
seemed  more  appropriate  to  a  more  extensive  IES  training  program  and  was  a',  so 
intentionally  left  out  of  experimentation.  The  small  sample  of  subjfcts  used 
in  this  study  was  a  shortcoming  which  could  not  be  prevented  due  to  rime, 
equipment  and  subject  availability  constraints.  More  subjects  and  mere  train¬ 
ing  sessions  per  subject  would  have  been  desirable.  However,  since  • uture 
experimentation  with  IES  hy  NAVTRAEQUIPCEN  appeared  to  be  a  realisti< 
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possibility,  it  was  considered  more  important  to  LSO  training  research  objec¬ 
tives  to  complete  this  study  and  report  its  results  within  the  contracted  cost 
and  time  frame- 


Results .  The  results  from  experimentation  with  IES  are  described  below.  Dis¬ 
cussion  of  recorded  performance  data  is  followed  by  discussions  of  question¬ 
naires,  then  the  performance  and  questionnaire  results  are  discussed  concur¬ 
rently.  In  view  of  the  extremely  small  sample,  tests  of  statistical  signifi¬ 
cance  were  not  performed. 

Overall  performance  measures  by  each  subject  improved  after  each  session. 
Measurement  of  performance  was  defined  during  analysis  by  the  percentage  of 
correct  calls  within  all  call  opportunities  (calls  made  plus  calls  which 
should  have  been  made.)  Figure  7  depicts  group  and  individual  performance 
over  the  four  sessions.  It  is  noteworthy  that  the  experienced  LSO  had  the 
highest  performance  scores  and  the  inexperienced  pilot  had  the  lowest,  which 
is  expected  in  a  valid  training  system.  Group  performance  data  was  also 
analyzed  from  several  other  perspectives. 

Figure  7  also  depicts  performance  by  types  of  deviations  over  the  four 
sessions.  Notice  that  LSO  performance  with  lineup  deviations  was  the  lowest, 
whereas  performance  for  AOA  and  combinations  of  deviations  tended  to  be  higher 
than  the  others.  High  performance  levels  for  combination  deviations  are 
probably  due  to  the  fact  that  a  single  call,  "waveoff,"  was  the  only  decision 
output  for  each  of  these  situations  in  this  study. 


Figure  7  also  depicts  performance  by  range  zone  over  the  four  sessions . 
Performance  during  "in  close"  tended  to  be  below  the  others  whereas  "at  the 
ramp"  performance  was  well  above  the  others  in  the  final  two  sessions.  The 
results  of  "at  the  start"  performance  are  probably  insignificant  since  there 
were  very  few  deviations  presented  within  that  range  zone.  There  are  several 
likely  reasons  for  poor  "in  close"  performance.  One  is  that  there  were  more 
"in  close"  situations  to  be  learned  (8  out  of  21  basic  training  scenarios). 
Additionally,  for  several  deviation  situations,  the  required  calls  differed 
significantly  from  "in  the  middle"  and  "at  the  ramp,"  making  "in  close"  range 
zone  estimation  a  more  critical  decision  factor  than  for  other  range  zones. 
inese  situations  are  delineated  below: 


Deviation 


In  the  Middle 


In  Close 


At  the  Ramp 


Low 
Slow 
Fast 
Hi  ght 
Left 


'you're  low" 

"You're  slow" 

"you're  fast" 

"you're  lined  up  right" 
"you're  lined  up  left" 


"power” 

"power" 

none 

"left  for  lineup" 
"right  for  lineup" 


"waveoff" 
"waveoff" 
"waveof  f " 
"waveoff" 
"waveoff" 


The  high  performance  levels  for  "at  the  ramp"  were  probably  influenced  by  the 
fact  that  for  any  significant  deviation  in  this  range  zone,  the  only  correct 
call  was  ’waveoff." 


From  the  perspective  of  calls  used,  performance  quality  with  the  "wave- 
off”  and  "you're  slow”  calls  was  dramatically  higher  than  for  any  others. 
"Waveoff"  call  performance  was  probably  influenced  by  the  heavy  training  em¬ 
phasis  placed  on  its  utilization.  Eight  of  the  21  basic  training  scenarios 
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elicited  the  "waveoff"  call.  Additionally,  many  of  the  complex  testing 
scenarios  were  designed  to  result  in  "waveoff"  situations. 

The  data  were  analyzed  to  see  if  performance  with  the  basic  training 
scenarios  portion  of  testing  differed  from  that  with  the  more  complex 
scenatios.  Surprisingly,  the  differences  appeared  inconsequential.  Data  was 
also  analyzed  to  see  if  there  were  differences  in  performance  between 
informative  and  imperative  calls.  The  differences  here  also  appeared  to  be 
insignificant. 

In  the  questionnaire,  subjects  were  asked  to  rate  how  well  they  could 
detect  the  various  deviations  within  the  various  range  zones.  Glideslope 
deviations  (high  in  particular)  were  rated  easiest  to  detect  and  angle  of  at¬ 
tack  deviations  (fast  in  particular)  were  rated  lowest.  The  subjects  ranked 
"in  close"  as  the  range  zone  in  which  deviations  were  easiest  to  detect.  The 
"middle"  and  "ramp"  range  zones  were  also  rated  fairly  high.  The  "start" 
range  zone  was  rated  quite  a  bit  lower  than  the  others,  as  was  expected.  The 
subjects  noted  that  it  was  difficult  to  perceive  deviations  due  to  the  small 
size  of  the  aircraft  image  at  long  ranges. 

After  each  session  the  subjects  were  asked  to  identify  the  two  decisions 
which  were  best  learned  during  that  session.  The  decision  to  call  "power"  for 
low  glideslope  in  close  was  most  frequently  noted  (four  times).  "Waveoff"  was 
called  for  high,  fast  in  close  and  for  low,  slow  in  close,  and  "you're  lined 
up  right/left"  was  called  for  lineup  deviations  at  the  start  (three  times 
each )  . 


To  help  identify  potential  shortcomings  in  IES  ef f ectivenss ,  the  subjects 
were  also  asked  to  identify  two  decisions  for  which  the  least  learning 
occurred  during  each  session.  "Waveoff"  for  lineup  deviations  at  the  ramp  was 
mentioned  most  frequently  (five  times).  "Right/left  for  lineup"  for  lineup 
deviations  in  close  was  also  mentioned  frequently  (four  times). 

At  the  end  of  the  study  the  subjects  were  asked  to  rate  how  well  they  had 
learned  the  various  decisions  in  the  course  of  the  study.  The  purpose  of  this 
question  was  to  force  the  subjects  to  reflect  upon  the  overall  learning 
experience  with  IES.  All  subjects  felt  that  they  had  "learned"  or  "learned 
very  well"  all  decisions,  with  one  exception.  One  subject  was  r.  it  sure  if  he 
had  learned  to  use  the  call  "waveoff"  for  lined  up  right  at  the  ramp.  His 
reason  was  based  upon  uncertainty  in  perceiving  the  deviation.  Decisions 
related  to  lineup  deviations  at  the  start,  in  the  middle  and  at  the  ramp,  and 
glideslope  deviations  at  the  start  received  the  lowest  overall  ratings. 

The  questionnaire  also  brought  out  several  aspects  of  IES  which  the  sub¬ 
jects  felt  needed  improvement.  Problems  with  speech  recognition  were  most 
frequently  noted.  From  the  subjects'  standpoint  it  caused  frustration.  It 
was  also  noted  that  automated  performance  measurement  was  degraded  and  that 
"instructor"  (system  operator)  loading  was  high  due  to  manually  recording 
voice  calls.  Difficulty  detecting  the  fast  AOA  deviation  was  also  noted  by 
all  subjects.  Two  of  the  subjects  commented  that  there  should  have  been  more 
"ok"  approaches  (no  deviations)  during  training  and  testing.  It  was  also 
suggested  that  the  training  portion  of  each  session  should  include  several 
complex,  multi-deviation  scenarios  such  as  those  presented  during  testing. 

The  authors  concur  that  these  suggestions  would  have  improved  the  reliability 
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of  study  results.  A  "pickle"  switch  was  suggested  by  two  subjects  One 
subject  suggested  a  large  display  and  another  was  critical  of  aircraft  image 
"jerkiness"  in  close.  One  subject  suggested  that  the  text  messages  would  be 
easier  to  monitor  if  located  nearer  to  the  aircraft  position  grid.  The 
authors  concur  that  the  suggested  modifications  to  IES  are  potential 
improvements,  but  their  cost-,  ffectiveness  is  questionable. 

The  experienced  LSO  had  tome  additional  critique  items  of  interest.  He 
felt  that  training  and  performance  evaluation  should  incorporate  some  pre¬ 
cautionary  calls  such  as  "check  your  lineup,"  "you’re  drifting  left,"  "don’t 
settle,"  etc.  These  would  help  provide  instruction  in  anticipatory  decision 
skills  in  addition  to  the  "snapshot"  skills  currently  addressed  by  IES.  It 
would  also  provide  finer  tuned  control  of  aircraft  dynamics  by  the  LSO  trainee 
since,  when  properly  us  3d,  precautionary  calls  can  preclude  gross  deviations. 
He  was  also  critical  of  the  roll  dynamics  associated  with  lineup  deviations, 
since  most  real-world  3 ineup  deviations  are  more  subtle.  Increased  feedback 
and  instructional  guidance  to  the  student  at  the  end  of  an  approach  was  also 
suggested  and  was  strongly  concurred  in  by  the  authors.  He  felt  that  the 
horizon  should  be  more  prominent  and  that  the  lineup  deviations  and  correction 
rates,  in  close  and  at  the  ramp,  were  excessive  departures  from  realism.  He 
also  felt  that  a  "ramp  strike"  should  be  one  of  the  possible  outcomes  of  a 
poor  approach. 

The  subjects  had  several  inputs  regarding  perceived  strengths  of  IES  for 
LSO  training.  All  agreed  that  IES  promotes  a  conceptual  understanding  of  LSO 
decision  skills.  They  felt  that  the  interactive  LSO  task  performance  aspect 
of  IES  is  a  valuable  introduction  to  waving,  since  it  allows  a  trainee  to  gain 
early  experience  with  the  key  elements  of  the  LSO's  decision  loop:  detecting, 
deciding  and  taking  action.  The  inexperienced  pilot  felt  strongly  that  his 
experience  with  IES  in  this  study  would  be  helpful  to  him  in  his  early 
on-the-job  LSO  training.  The  aircraft  position  grid,  in  conjunction  with  the 
"snapshot"  approach  parameters  associated  with  voice  calls,  encourage  him  to 
approach  learning  the  LSO  job  from  a  "window"  concept.  He  said  that  he  will 
establish  a  series  of  spatial  "windows"  which  he  will  use  to  guide  his 
learning  of  when  to  use  the  various  LSO  voice  calls.  The  experienced  LSO  also 
suggested  that,  if  situations  requiring  a  larger  selection  of  voice  calls  were 
incorporated  into  IES  training,  this  systen  could  help  promote  voice  call 
standardization  in  the  LSO  community.  A  quote  from  the  experienced  LSO  is 
also  noteworthy  in  providing  insight  into  the  potential  value  of  LSO  training 
system  support  to  the  LSO  training  program:  "...  the  machine  is  extremely 
valuable.  What  it  It cks  in  reproducing  reality,  it  more  than  makes  up  for  by 
providing  student  LSf/s  valuable  decision  making  training  which  is  never  avail¬ 
able  in  sufficient  quantity  in  the  fleet.  It  will  eliminate  the  necessity  of 
the  Air  Wing  LSO  con  -entrating  on  basics...." 

There  were  some  interesting  correlations  between  task  performance  a  id 

questionnaire  data.  Of  major  interest  to  LSO  training  goals  is  task  perform¬ 
ance  within  the  "in  close"  range  zone.  Pilot  and  LSO  errors  here  can  lead  to 
tragic  landing  results.  "In  close"  was  the  range  zone  in  which  the  poorest 
performance  was  demonstrated,  whereas  the  subjects  felt  that  deviation  detec- 
tio  i  was  easiest  in  this  area.  As  mentioned  earlier,  the  number  and  cou¬ 
ple  ;ity  of  the  "in  close"  situations  may  have  adversely  affected  performance. 
As  figure  7  (presented  earlier)  shows,  "in  close"  performance  appeared  to  be 
dramatically  improving  in  the  final  session.  This,  coupled  with  the  subjects' 
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opinion  of  their  detection  skills,  could  indicate  that  there  were  too  few 
training  sessions  to  fully  integrate  the  decision  and  action  aspects  of  the 
task  with  detection.  AOA  was  rated  by  the  subjects  as  the  most  difficult  type 
of  deviation  to  detect,  but  performance  with  AOA  deviations  was  well  above 
average. 

Regarding  how  well  the  subjects  learned  to  "wave"  various  situations 
there  were  also  some  noteworthy  correlations.  Performance  was  very  high  for 
situations  consisting  of  combination  deviations.  This  compares  favorably  with 
the  perception  of  the  subjects  that  low,  slow  in  close  and  high,  fast  in  close 
(two  situations  leading  to  the  "waveoff”  call)  were  well  learned  during  IES 
training.  This  is  probably  indicative  that  the  subjects  easily  grasped  the 
decision  rule  promoted  by  the  training  sessions  that  two  major  deviations  in 
close  require  a  "waveoff"  call.  Conversely,  although  the  subjects  felt  that 
they  learned  well  the  situation  low  in  close  (leading  to  the  "power"  call), 
their  performance  for  this  was  below  average.  This  seems  to  indicate  the 
difficulty  subjects  had  in  discriminating  voice  call  requirement  transitions 
by  range  for  low  deviations  during  approach,  as  discussed  earlier.  Situations 
consisting  cf  lineup  deviations  in  close  and  at  the  ramp  were  considered  most 
difficult  to  learn  by  the  subjects.  Performance  data  for  lineup  deviations  in 
close  concurred  with  this  perception.  However,  data  for  lineup  deviations  at 
the  ramp  indicated  a  high  level  of  performance. 

In  summary,  several  results  of  IES  utilization  were  encouraging.  There 
was  a  definite  imporvement  in  subjects'  ability  to  perceive  deviations  from 
the  original  version  of  IES-  The  instructional  feed  back  text  feature  of  IES 
was  also  a  definite  improvement.  Even  though  performance  was  less  than 
expected,  the  learning  progres  over  the  four  sessions  was  encouraging.  The 
high  level  of  performance  and  rate  of  learning  for  use  of  the  "waveoff"  call 
were  particularly  encouraging.  The  most  encouraging  result  of  all  was  the 
subjects'  high  level  of  receptivity  to  the  automated  LSO  training  system  con¬ 
cepts  of  interactive,  decision-oriented  training.  The  fact  that,  after  four 
training  sessions,  subjects  were  making  correct  calls  only  about  fifty  percent 
of  the  time  was  disappointing.  Poor  student  performance  for  in  close  situ¬ 
ations  and  poor  system  speech  recognition  performance  were  also  disappointing 
results.  In  retrospect,  the  authors  feel  that  the  subjects  should  have 
received  more  taining  sessions  to  better  assess  student  learning  progress. 

for  a  field  application  there  are  several  signifcant  snortcommgs  to  IES. 
Operation  of  IES  was  not  designed  for  "turn  key"  utilization  in  a  field 
training  setting.  The  scope  of  instruction  for  field  utilization  is  extremely 
limited,  only  encompassing  minimal  interactive  LSO  decision  situations  and 
voice  calls.  Flexibility  for  scenario  revisions  and  expansion  is  very  limited 
due  to  hardware  constraints  and  software  structure  rigidity.  Resolution  of 
system  shortcomings  is  possible  with  new  hardware  and  revised  software,  but 
there  is  probably  a  significant  cost  consideration. 

AWAVS  EXPERIMENTATION 

Experimentation  was  planned  for  the  Aviation  Wide  Angle  Visual  System 
(AWAVS)  at  NAVTRAEQUIPCEN  to  investigate  visual  simulation  requirements  for  an 
LSO  training  system.  Subsequent  paragraphs  briefly  describe  AWAVS,  experi¬ 
mental  plans,  experimentation  activities  and  results.  An  expanded  discussion 
of  this  activity  is  available  in  Appendix  C. 
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AWAVS  is  a  large  iomed  visual  simulation  surrounding  an  aircra  rt  cockpit 
(^-2).  The  aircraft  s  mulation  includes  a  motion  base.  AWAVS  can  project 
ir  tegrated  imagery  to  he  display  area  (about  180°)  through  two  projectors, 
oi.  ?  for  background  scene,  the  other  for  target  image.  Projected  imagery  can 
cone  from  either  a  model  board  or  computer  generation  source.  AWAVS  was 
designed  for  research  with  visual  simulation  requirements  for  pilot  training. 
Fo;  this  experiment  a  computer  generated  A-7  aircraft  data  base  wai  developed 
an.  a  model  aircraft  carrier  deck  was  constructed.  AWAVS  projectec  the 
carrier  image  through  the  background  projector  aid  the  A-7  image  through  the 
target  projector  to  provide  a  carrier  approach  scene  from  the  LSO's 
perspective. 

Oric inal  experimental  plans  called  for  investigation  of  several  visual 
simulation  variables  including  resolution,  field  of  view  and  aircraft  image 
level  of  detail.  Testing  was  to  be  done  under  simulate  1  day  and  night  ambient 
light  conditions.  Due  to  system  capability  constraints,  plans  were  reduced  to 
looking  only  at  field  of  view  variations,  and  only  under  simulated  day  condi¬ 
tions.  Constraints  included:  limited  preprogrammed  control  of  aircraft 
approach  dynamics,  no  night  aircraft  image,  and  only  a  single  image  level  of 
detail  available.  Additionally  it  turned  out  that  only  the  highest  system 
resolution  allowed  recognition  of  the  aircraft  image.  The  question  actually 
addressed  was  whether  f  Leld  of  view  size  made  a  difference  in  LSO  perception 
and  voice  call  performs  ice.  This  was  considered  important  since  field  of  view 
size  is  a  significant  cost  consideration  for  candidate  visual  systems. 

In  conducting  the  experiment,  six  highly  skilled  LSCs  were  used  as  sub¬ 
jects.  Canned  approach  profiles  were  displayed  to  the  subjects  who  were  in¬ 
structed  to  make  LSO  calls  as  if  they  were  "waving"  the  aircraft.  Various 
sizes  of  glideslope  and  lineup  deviations  were  included  in  most  of  the  pro¬ 
files.  Some  profiles  had  no  deviations.  Only  a  portion  of  each  approach  was 
shown  because  the  image  could  only  be  displayed  while  within  a  limited  area 
defined  by  the  position  of  the  fixed  target  projector.  The  start  ranges  for 
the  approaches  varied  between  about  4000  and  2000  feet  from  the  ideal  touch¬ 
down  point.  Each  approach  terminated  on  LSO  call  or  at  about  1500  feet  if  no 
call  were  made  by  that  range.  Glideslope  and  lineup  deviations  varied  between 
large,  medium,  small  and  none.  If  no  call  were  made  by  the  subject  during  an 
approach,  he  was  asked  whether  he  perceived  any  deviation  at  termination.  At 
the  end  of  the  study,  subjects  filled  out  questionnaires  and  wt;e  interviewed 
by  another  senior  LSO  who  assisted  in  the  experiment. 

Analysis  of  the  experimental  data  in  Appendix  B  revealed  that  field  of 
view  size  had  a  statistically  significant  effect  on  LSO  performance  in  detect¬ 
ing  deviations,  but  not  on  making  calls.  It  appears  from  inspection  of  Figure 
B-4  in  Appendix  B  that  best  LSO  performance  was  with  the  medium  size  field  of 
view.  There  were  quite  a  few  incorrect  detections  by  the  subjects.  Some  were 
in  terms  of  false  alarms  (saying  there  was  a  deviation  when  none  existed). 

The  overall  probability  of  a  false  alarm  was  over  30  percent.  Others  involved 
errors  in  the  direction  of  deviations  (high  versus  low  and  right  versus  left) . 
The  glideslope  error  rate  was  five  percent,  and  lineup  was  three  percent. 

Several  confounding  aspects  of  the  experiment  were  noted  by  the  authors, 
some  of  which  were  confirmed  through  questionnaire  and  interview  responses. 

The  LSO  task  of  the  subjects  was  somewhat  unrealistic  in  that  aircraft  flight 
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dynamics  did  not  include  roll  or  attitude  changes.  Deviations  were  smoothly 
depicted  by  a  straight  line  vector.  Both  factors  were  caused  by  system  con¬ 
straints  in  the  control  of  aircraft  dynamics.  Tuei/c  were  criticisms  of  LSO 
task  context  in  that  the  approach  did  not  continue  to  touchdown,  thus  pre¬ 
cluding  approach  results  feedback,  and  there  was  no  engine  noise.  There  were 
strong  criticisms  of  aircraft  image  quality  (resolution)  and  the  "fuzziness" 
of  the  background  scene.  The  size  of  the  aircraft  image  relative  to  range  was 
questioned  by  the  subjects  (they  felt  it  was  too  small).  However,  a  double¬ 
check  of  dimensions  confirmed  the  sizing  to  be  correct.  In  rating  realism,  on 
a  scale  of  1  to  5  (where  1  =  unrealistic  and  5  =  veiy  realistic)  the  back¬ 
ground  scene  was  rated  at  slightly  less  than  3  and  the  aircraft  image  was 
rated  at  3.  There  were  also  complaints  about  the  small  time  frame  of  several 
approach  profiles.  Some  were  as  short  as  3  -  5  seconds  prior  to  termination. 

There  were  several  other  noteworthy  comments  from  the  subjects.  Only 
half  of  the  subjects  felt  that  field  of  view  variations  affected  their  ability 
to  wave  the  approaches.  The  subjects  were  also  asked  to  rate  the  adequacy  of 
the  field  of  view  variations  for  LSO  training.  On  a  scale  of  1  to  5  (where 
1  =  inadequate  and  5  =  adequate),  wide  was  rated  at  4.5,  medium  at  4.2  and 
narrow  at  3.2.  There  were  no  "inadequate"  ratings  given  by  the  subjects.  A 
few  of  the  subjects  who  had  seen  the  LSO  Reverse  Display  felt  that  AWAVS  image 
resolution  was  significantly  poorer.  However,  they  liked  the  openness  of  the 
AWAVS  display  area  which  permits  two  LSOs  to  view  the  scene.  They  felt  that 
this  was  an  advantage  for  LSO  team  training.  All  subjects  agreed  that  a 
sophisticated  visual  simulation,  as  demonstrated  by  AWAVS  and  the  LSO  Reverse 
Display,  would  be  benefit ial  to  LSO  training.  This  is  in  agreement  with  Hooks 
and  McCauley. 3.  with  regard  to  an  LSO  training  system  application,  the  size 
of  a  projection  system  like  AWAVS  appears  to  be  a  potential  disadvantage  in 
terms  of  facility  requirements. 

The  implication  of  the  data  and  commentary  provided  in  the  paragraphs 
above  is  that  the  experiment  did  not  successfully  answer  the  question  of 
visual  system  field  of  view  requirements.  However,  the  experience  of  working 
with  AWAVS  leads  the  authors  to  feel  that  its  projection  system  is  less  desir¬ 
able  than  a  direct-view  CRT  system  such  as  that  incorporated  in  the  LSO 
Reverse  Display.  The  primary  factors  in  this  opinion  are  the  resolution 
limitations,  relatively  high  procurement  costs  and  extensive  facility  require¬ 
ments  of  a  large-screen  projection  system. 


3.  J.T.  Hooks  and  M.E.  McCaulley,  Training  Characteristics  of  LSO  Reverse 
D i splay ,  Technical  Report  79-C-0101-1,  NAVTRAEQUIPCEN ,  (in  press). 


34 


NAVTRAEQUI PCEN  78-C-O 151-1 


SECTION  IV 
CONCLUSIONS 

The  authors  conclude  that  an  automated  LSO  training  system  is  a  fe  isible 
concept.  The  term  "automated  LSO  training  system1'  implies  a  stand-alor,  ■ 
interactive  system,  consisting  of  high  fidelity  night  visual  simulation, 
automated  speech  recognition  and  software  control  of  instructional  situations. 
It  does  not  imply  an  automated  adaptive  capability;  that  question  remains 
unresolved.  Subsequent  paragraphs  of  this  section  discuss  this  and  other 
conclusions  which  evolved  from  this  study.  The  initial  discussions  address 
general  LSO  training  system  concepts  and  the  final  discussions  address  the 
laboratory  system. 

LSO  TRAINING  SYSTEM  CONCEPTS 

Support  for  the  automated  LSO  training  system  concept  mentioned  above  is 
based  on  several  factors.  It  appears  that  IES  utilization  has  proved  the 
capability  of  LSO  training  system  concepts  for  basic  skill  acquisition.  The 
results  of  LSO  Reverse  Display  evaluation,  as  reported  by  Hooks  and 
McCauley^,  appear  to  support  LSO  training  system  benefits  to  higher  level 
still  acquisition  such  as  "waving"  under  pitching  deck  and  Manually  Operated 
Visual  Landing  Aid  System  (MGVLAS)  conditions.  Though  IES  was  a  relatively 
crude  representation  of  a  sop  listicated  automated  LSO  training  system,  there 
were  positive  indications  of  training  effectiveness  potential  in  this  study: 
receptivity  of  subject?  to  its  interactive  decis ion-or rented  training  merits 
and  their  task  perform.- ice  improvements  in  a  limited  training  period,  espe¬ 
cially  in  their  grasp  of  when  to  use  the  "waveoff"  call.  From  a  system  devel¬ 
opment  standpoint,  a  potentially  effective  level  of  training  capability  was 
produced  in  a  very  constrained  developmental  environment  (hardware  and  cost 
limitations).  Therefore,  greatly  improved  system  effectiveness  over  that  of 
IES  is  a  reasonable  expectation  for  a  normal  training  system  development  situ¬ 
ation.  As  reported  by  Hooks  and  McCauley,  an  adequate  visual  simulation  capa¬ 
bility  has  already  been  s  .ccessfully  demonstrated  in  the  LSO  Reverse  Display. 
Although  experimentation  \  ith  AWAVS  did  not  resolve  visual  simulation  require¬ 
ment  uncertainties,  it  die  provide  insight  into  some  of  th 5  limitations  of  a 
projection  visual  system  to  an  LSO  training  application.  The  authors  also  feel 
that  it  is  reasonable  to  anticipate  that,  with  adequate  compute..'  resources, 
software  control  of  situation  presentation  and  LSO  interaction  can  reach 
significantly  higher  instructional  levels  than  those  provided  by  IBS. 

Although  speech  recognitic n  difficulties  were  encountered  with  IES,  the 
problems  (non-  optimized  threshold  parameters  and  absence  of  context  VDC )  have 
been  identified  and  their  resolution  is  feasible. 

Based  upon  this  study,  as  well  as  pre  iminary  results  from  the  LSO 
Reverse  Display  evaluation,  there  appear  t>  be  several  fu  -ctional  aspects  of 
an  operational  automated  LSO  training  syst  :m  which  are  ne-  dea  for  training 
effectiveness : 

•  provisions  for  LSO  task  interactioi 


4.  J.T.  Hooks  and  M.E.  McCauley,  Training  Charact  enst  cs  of  LSO  Reverse 
Display,  Technical  Report  79-C-0101-1,  NAVTRAEQUI PCEN ,  (  n  press). 
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•  night  carrier  visual  approach  scene  simulation 

•  simulation  of  multiple  aircraft  types 

•  pitching  deck  simulation 

•  MOVLAS  training  capability 

•  "canned"  or  interactive  preprogrammed  approach  scenario  provisions  to 
support  specific  learning  goals 

•  graphic  cueing  aid(s) 

There  are  also  several  functional  aspects  of  an  operational  automated  LSO 
training  system  which  remain  questionable.  Some  of  the  more  important 
include: 

•  day  Carrier  visual  approach  scene  simulation 

•  automated  adaptive  syllabus  control 

•  automated  performance  evaluation 

•  background  sound  simulation 

•  visual  system  field  of  view  size 

Later  this  report  will  present  recommendations  regarding  resolution  of 
these  and  other  system  design  uncertainties  through  the  development  and 
testing  of  a  prototype  automated  LSO  training  system.  Appendix  D  delineates 
the  recommended  featues  and  capabilities  for  the  prototype  system,  several  of 
which  are  intended  to  aid  in  the  resolution  of  operational  system  design 
uncertainties. 

Based  on  study  results,  there  also  appear  to  be  several  LSO  training  sys¬ 
tem  alternatives  which  singularly,  or  within  a  "family  of  training  systems” 
concept,  can  increase  LSO  training  program  effectiveness.  Table  6  outlines 
the  alternatives.  The  paragraphs  below  describe  envisioned  capab  lines, 
probable  limitations  and  the  estimated  level  of  LSO  skill  acquisition  for 
which  each  system  is  appropriate. 

Alternative  1  is  a  demonstration  system  incorporating  trie  dynamic  presen¬ 
tation  of  approach  situation  scenes,  possibly  tnrough  movies,  within  a  crudely 
simulated  LSO  workstation.  This  would  essentially  provide  job  familiarization 
to  LSO  trainees  and  would  be  a  tool  appropriate  on.y  to  Phase  I  traini.ig. 
Alone,  it  would  have  only  minimal  impact  on  LSO  training  program 
effectiveness. 


A  t' :  r  rat  .  ve  II  i an  interactive  part  task  training  system,  r. malar  to 
the  labor  a  tor-/  L30  ‘ram:  rg  system  (1ES,  developed  during  this  study,  but  with 
enhanced  capabilities  Trie  enhancements  would  consist  of  improved  speech 
recognition  and  ----- forma  nee  feedoack  plus  an  increase  in  the  number  of 
approach  situations  presented  and  their  related  voice  calls.  The  instruc¬ 
tional  orientation  of  this  device  would  be  toward  basic  LSO  decision-making 
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I  —  Demonstration: 

II  _  Part  Task: 

III  _  High  Fidelity  Part  Task: 

IV  —  Modified  LSO  Reverse  Display: 

V  -  Universal  LSO  Trainer: 


VI  _  Universal  Adaptive  LSO 
Training  System: 
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SYSTEM  ALTERNATIVES 

non- interactive 
approach  scene  "movies" 
work  station  representation 
LSO  job  familiarization 

interactive 

enhanced  version  of  laboratory  LSO 
training  system 
basic  decision  skills 

interactive 

enhanced  laboratory  ISO  training 
system 

basic  decision  skills 
basic  p  :rceptual  ski  Is 

interac live 

additional  canned  approaches 
multiple  aircraft  t /pes 
instructor  control  of  aircraft  with 
"joystick" 

Phase  III  LSO  training  support 

interactive 
night  scene 

multiple  aircraft  types 
computer  control  of  situations 
speech  recognition 
Phase  III  and  refre^ner  training 
support 

interactive 
night  scene 

multiple  aircraft  types 
computer  control  of  situa  ions 
speech  recognition 
performance  evaluation 
adaptive  logic 
instructor less  training 
Phase  III  and  refresher  training 
support 
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jfti  !  is;.  A.  .ru*,  this  system  should  have  a  positive  impact  on  LSO  tuning  pro- 
gr-un  effectiveness  by  minimizing  instructional  attention  to  decision-making 
basics  during  on-the-job  training  (GJT).  It  should  also  be  an  effective  Phase 
I  training  complement  to  the  demonstration  system  mentioned  earlier. 

A'.  t  creative  I  IT  is  ai  so  an  interactive  part  task  training  system.  The 
"Jijor  improvement  over  Alternative  II  is  the  incorporation  of  a  high  fidelity 
night  visas L  simulation  similar  to  that  of  the  LSO  Reverse  Display.  A  single 
type  of  aircraft,  such  as  the  A7 ,  would  be  ro  il istical ly  simulated,  thus 
a’: owing  instr  actional  attention  to  percept ua 1  as  well  as  decision  skills. 

,:h;  .  s/stf-n  would  be  beneficial  to  Phase  I  training.  If  located  at  multiple 
fleet  sites,  it  should  also  provide  limited  s  .pport  •.  Phase  II  and  Phase  III 
training.  It  would  also  have  sufficient  capabilities  to  support  additional 
research  into  LSO  training  system  concepts. 

Alternative  IV  is  a  modified  version  of  i ne  Ls»0  Reverse  Display.  LSO 
vase  r.  -action,  with  the  pilot  flying  the  A'.'s  Night  Carrier  Lancing  Trainer 
.  T .  •  v-L:  be  retained.  However,  instructional  control  of  situation  presen¬ 

tation  would  be  enhanced  througn  an  increase  m  the  number  of  canned 
approaches  available  and  through  instructor  control  of  aircraft  dynamics  with 
a  "joystick."  The  scope  of  training  would  be  noreas  d  through  the  simulation 
of  multiple  typos  of  aircraft.  This  system  should  pro  ride  effective  instruc- 
:  icr.al  support  cor  a  significant  amount  of  Phase  III  LSO  training.  It  may 
a. so  prove  beneficial  in  refresher  LSO  training. 

Alternative  V  is  an  automated,  interactive,  stand-alone  and  universal  LSO 
trainer.  It  would  provide  a  r.ight  carrier  approach  scene  similar  in  fidelity 
to  LSO  Reverse  Display.  Other  functional  similarities  to  Alternative  IV 
..-.elude  multiple  aircraft  simulation,  "joystick"  and  computer  c  ir.tr  ol  of  situ¬ 
ations  and  speech  recognition.  Since  it  would  be  an  original  development, 
with  more  r*  cent  technologies  than  those  in  the  LSO  Reverse  Display,  the 
..erai'.  i  ;  fee  t. -res.  of  the  system  would  be  more  responsive  to  LSO  training  and 
fist  ructa.*  '  unction  requirements  based  on  "lessons  learned"  from  LSO  Reverse 
I uplay  util .cation.  This  system  should  support  Phase  III  and  refresher  LSO 
-raining  and  would  likely  support  limited  " instructorless"  LSO  trainee 
practice  exercises.  It  would  also  support  LSO  training  research  due  to  its 
data  collection  features. 

f.lternativ  •  '.'I  is  an  expansion  of  Alternative  V  to  include  automated  per- 
:  ort.iar.ee  eva  !>.«.. ..c-n  and  adaptive  syllabus  control.  These  features  would 
reduce  Instructor  loading  and  permit  instructor less  training  for  selected  seg¬ 
ments  of  LSO  train. ng.  Anticipated  limitations  to  the  scope  of  instructor  less 
••.raining  are  bur-ai  on  current  uncertainties  regarding  knowledge  o;  va.ic.  per¬ 
formance  meaner -s  L.roughout  the  range  of  LSO  ski. is,  arid  the  improver,  status 
adu.t. ve  instructional  techniques  for  complex  skills,  such  as  those  of  the 

i..-:  ..  This  sy.tetn  shou.d  provide  support  to  Phase  III  an.  refresher  LSO  train¬ 
ing  ,  It  shoal  1  !:.«>  -T.n  an  excellent  tool  for  reseax  r.  into  LaO  t.  p.  •  •- 

,  r-.c.  ■:  learn i  .  str«.  •  y.  .  This  . .  •  i‘  for.  .  t 

>.  ’  .i  .••  i  -..-.id it.-  us  an  experimental  r.rct c type  l.LO  train  .ng  system  *  t  ;h 

i'  i  k.r  i s  i  v.v-maki  ng  gutd..r.ce  for  wnvther  more  systems  are  needed  and 

■  or  what  charactvrist.  :s  are  really  required. 

There  a.,  uuve.-ai  factors  related  to  decisions  regard. ng  the  procurement 
>2  i.SO  train ; ng  sy  ; ferns  such  as  those  described  above.  Subsequent  paragraphs 
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iiscuss  tr.o  factors  .  -nt  i  f  ied  in  the  course  of  this  study.  The  :  actor:  are 
also  outline  .  n.  Tai.lt  7. 

TA1  -E  7.  PROCUREMENT  DECISION  FACTORS 

Cost:  development 

facilities 
maintenance 
student  population 

Training  ef  ectiveness:  acceleration  of  LSO  training 

accident  prevention 

siting 

timing  of  utilization 
travel  funding 

training  program  management 
instructor  availability 
user  attitudes 

One  of  the  major  concerns  of  high-level  Navy  personnel  is  LSO  training 

system; s)  cost.  A  highly  sophisticated  automated  LSO  training  system  has  a 
multi-million  dollar  cost  potential  just  for  initial  procurement.  This 
includes  system  design,  development,  testing,  as  well  as  availability  of 
adequate  supporting  facilities.  Additionally,  there  are  long  term  maintenance 
cost  considerations.  This  includes  maintenance  of  system  operating  capabili¬ 
ties  plus  periodic  update  of  system  training  capabilities  and  training  program 
plans  in  response  to  changing  L SO  training  needs.  Concern  has  been  expressed 
for  the  costs  mentioned  above,  in  view  cf  the  small  size  of  the  Navy's  LSO 
population.  There  are  approximately  300  LSO  billets  in  the  Navy,  of  which 
about  one-third  (90-100)  are  filled  with  trainees  who  are  working  toward  the 
productive  skili  level  of  Wing  LSO. 

In  contrast  to  cost  concerns  are  considerations  for  LSO  training  program 
effectiveness.  Shortages  of  skilled  LSOs  continue  to  exist  in  the  fleet. 

Pilot  retention,  curtailed  carrier  operations,  and  the  lengthy  and  inefficient 
LSO  training  process  continue  to  be  the  major  causes  o:  this  d  ficiency. 
Concerns  continue  to  be  raised  regarding  the  actual  skill  levels  cf  experi¬ 
enced  LSOs,  many  of  whom  have  had  very  limited  exposure  to  demanding  aspects 
of  the  LSO  job  such  as  pitching  deck,  MOVLAS,  aircraft  malfunctions  and 
stressful  operational  situations.  Only  the  LSOs  operating  aboard  the  USS 
Midway,  stationed  in  Japan,  have  the  opportunity  for  continual  exposure  to 
extensive  carrier  landing  operations  over  a  three-year  tour  of  sea  duty. 

Others  are  expose:,  to  lengthy  periods  of  inactivity  between  deployments  and 
carrier  landing  operations  while  deployed.  An  LSO  training  system  can  supple¬ 
ment  on-the-job  training  with  interactive,  instructionally-control .ed  "waving" 
experience.  The  authors  and  many  experienced  LSOs  feel  that  such  a  system  has 
the  potential  to  accelerate  basic  skill  acquisition,  th  ;s  enabling  a  trainee 
to  "get  the  pickle”  earlier  in  his  shipboard  training  pj  ogram.  The  potential 
for  increased  experience  with  complex  waving  situations  is  another  positive 
factor  in  support  of  an  LSO  training  system.  Improvements  in  carrier  landing 
safety  appear  to  be  potential  benefits  frc m  effective  LSO  training  system 
ut  1 1  izatio.i . 


Accessibility : 


Utilization: 
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Accessibility  to  an  LSO  training  system  is  a  practical  consideration 
which  has  significant  cost  and  training  effectiveness  implications.  Limited 
access  to  a  training  system,  regardless  of  its  effectiveness,  minimizes  its 
value.  Ir.  the  case  of  LSO  training,  timeliness  of  LSO  training  system 
utilization  would  be  a  very  important  training  program  effectiveness  factor. 
The  trainee  should  have  access  to  the  training  system  during  FCLP  work-ups  and 
in  close  tine  proximity  to  carrier  operations,  in  order  to  maximize  transfer 
of  skill  acquisition  between  simulated  and  actual  operating  environments.  The 
LSO  popular. .  on  in  the  Navy  has  a  wide  geographical  spread,  encompassing  five 
separate  fl-.-et  population  centers,  as  well  as  several  Naval  Aviation  Training 
Command  locations.  Limiting  the  locations  of  an  LSO  training  system,  while 
keeping  procurement  costs  down,  causes  difficulty  in  providing  timely  access 
for  some  trainees  and  increases  travel  funding  requirements  for  system 
utilization  Additionally,  there  are  LSO  hardships  to  be  considered.  LSOs 
typically  s^end  more  time  away  from  home  during  FCLP  and  carrier  work-up 
periods  than  other  fleet  squadron  pilots.  Over  two-  thirds  of  all  LSO  billets 
and  nearly  all  trainees  operate  from  five  fleet  areas:  Norfolk,  Jacksonville, 
San  Diego,  Lemoore  and  Whidbey  Island.  Of  these,  the  Norfolk  (NAS  Norfolk, 

NAS  Oceana),  San  Diego  (NAS  Miramar,  NAS  North  Island)  and  Jacksonville  (NAS 
Cecil  Field)  have  the  largest  LSO  populations  and  therefore  appear  to  be  the 
most  promising  locations  for  LSO  training  systems,  if  limited  systems  were  to 
be  procured.  Although  Norfolk  (about  70  LSOs)  and  San  Diego  (about  60  LSOs) 
have  higher  populations  than  NAS  Cecil  Field  (about  50  LSOs),  the  LSO  Phase  I 
School  at  NAS  Cecil  Field  is  a  favorable  factor  for  that  location.  On  the 
other  hand,  NAS  Cecil  Field  already  has  the  LSO  Reverse  Display  device  for 
limited  support  to  LSO  training.  Procurement  costs  for  five  systems  would  be 
higher  than  for  a  single  system,  or  for  one  system  on  each  coast,  but  the 
tradeoffs  between  cost  and  training  effectiveness  are  not  as  clear. 

Another  major  consideration  is  training  system  utilization.  A  key  factor 
m  effective  utilization  would  be  training  program  management.  The  primary 
roles  of  management  would  include  encouragement  (or  direction)  of  appropriate 
rates  of  utilization,  monitoring  of  training  system  effectiveness,  and 
implementation  of  training  program  and  system  revisions  responsive  to  fleet 
needs.  The  recently  established  position  of  LSO  Training  Model  Manager  for 
the  Phase  I  School  Officer-in-Charge  provides  a  vehicle  for  effective  program 
management.  The  availability  and  motivation  of  instructor  LSOs  to  conduct  LSO 
training  system  instruction  are  other  factors  in  effective  system  utilization. 
Air  Wing  Staff  LSOs  must  be  encouraged  to  utilize  the  system  as  an  integral 
part  or  their  training  programs.  Overall  user  (LSO  and  trainee)  acceptance 
of  LSO  training  system  concepts  is  another  important  factor  in  effective 
rsyatem  utilization.  Since  LSO  trainees  typically  are  very  highly  motivated 
individuals,  their  altitudes  toward  a  training  system  will  be  strongly 
influenced  by  the  attitudes  of  experienced  LSOs.  The  positive  receptivity  of 
trainees  to  instruction,  coupled  with  positive  instructor  attitudes,  aru  very 
important  ingredients  to  LSO  training  system  effectiveness.  Thus,  promotion 
of  positive  user  attitudes  is  an  important  system  procurement  consideration. 
Development  and  thorough  testing  of  a  prototype  LSO  training  system  would  be 
an  important  step  in  building  training  system  credibility  for  the  LSO 
community . 
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The  authors  tentatively  conclude  that,  with  significant  enhancement,  the 
laboratory  LSO  traininc  system  could  be  a  beneficial  addition  to  Phase  I  and 
Phase  II  LSO  training  as  a  part  task  training  system.  To  meet  this  expecta¬ 
tion  improvements  woul>.  be  needed  in  speech  recognition,  scope  of  decision 
skill  coverage  by  training  scenarios  and  performance  evaluation,  and  ease  of 
instructor  operability.  The  primary  support  for  this  conclusion  was  the  sub¬ 
jects'  high  level  of  receptivity  to  the  automated  LSO  training  system  concepts 
of  interactive,  decision-orientated  training.  Additional  experimental  utili¬ 
zation  of  the  laboratory  LSO  training  system  would  be  required  for  confirma¬ 
tion  of  its  potential  value. 

The  authors  feel  that  this  part  task  training  concept  falls  far  short  of 
the  training  needs  of  the  LSO  community.  Thus  it  should  primarily  be  viewed 
as  a  relatively  low-cost  alternative  to  the  prototype  automated  LSO  training 
system  discussed  earlier. 
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SECTION  V 
RECOMMENDATIONS 

Recommendations  resulting  from  this  study  are  concerned  with  a  proposed 
prototype  LSO  training  system,  its  features  and  its  utilization.  Research  to 
date  has  been  unable  to  quantitatively  justify  an  automated  LSO  training  sys¬ 
tem  as  a  cost-effective  improvement  to  LSO  training.  Based  on  the  results  of 
developing  and  exercising  a  laboratory  LSO  training  system,  the  concept  has 
proved  feasible.  Support  for  the  concept  from  LSOs  during  this  and  other 
NAVTRA5QUIPCEN  research  programs  indicate  that  such  a  concept  can  have  strong 
positive  impact  on  reducing  the  time  required  for  LSO  skill  acquisition  and  on 
increasing  the  level  of  skill  acquired.  Thus,  it  is  recommended  that  a  proto¬ 
type,  automated  LSO  training  system  be  developed  for  experimental  validation 
and  refinement  of  this  concept,  while,  at  the  same  time,  providing  support  to 
the  LSO  training  program. 

Trie  results  of  prototype  system  utilization  should  provide  valid  reso¬ 
lution  to  several  major  areas  of  uncertainty: 

a.  Is  an  automated  LSO  training  system,  a  cost-effective  enhancement  to 
LSO  training? 

b.  What  features  are  actually  required  in  such  a  system? 

c.  How  should  the  system  be  employed  for  most  effective  support  of  LSO 
training? 

d.  How  many  systems  are  needed,  and  where  should  they  be  located? 

Additionally,  the  authors  feel  that  an  automated  LSO  training  system 
design  oriented  to  the  questions  above  would  also  be  useful  for  continued 
research  into  future  applications  of  automated  speech  technology  and  adaptive 
instructional  concepts.  Relevant  issues  include  performance  evaluation  for 
complex  speech-oriented  jobs,  instructorless  training,  and  perceptual  and 
decision  skill  acquisition. 

Associated  with  the  LSO  training  system  questions  presented  above,  are 
several  recommended  specific  lines  of  inquiry  to  be  addressed  by  the  prototype 
system.  The  question  of  >st-ef f ectiveness  must  first  include  a  study  of  pro¬ 
totype  LSO  training  system  effectiveness.  The  subjective  assessments  of 
potential  training  benefits  for  the  LSO  training  system  concept,  which  have 
resulted  from  this  study,  must  he  objectively  confirmed  prior  to  subsequent 
studies.  A  study  of  training  transfer  from  the  prototype  system  to  FCLP 
operations  or  preferably,  to  the  carrier  environment,  is  the  recommended 
course  of  action  for  this  confirmation.  In  conjunction  with  such  a  study,  the 
levels  of  LSO  skill  acquisition,  for  which  the  training  system  concept,  can  be 
applied,  must  be  validated.  This  is  necessary  in  order  to  ensure  relevant 
instructional  orientation  for  subsequent  lines  of  inquiry.  If  training  system 
def it lances  are  uncovered,  the  system  or  the  methods  of  its  am;  .oyment  must  be 
successfully  modified  prior  to  the  pursuit  of  any  other  lines  o'  inquiry. 

With  regard  to  potential  requirements  for  an  operational  LSO  training 
system,  there  are  several  recommended  areas  of  investigation.  One  area  to  be 
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addressed  is  the  potential  interference  to  speech  recognition  from  engine  and 
background  noise  in  the  student  station  of  an  LSO  training  system.  Another  is 
to  determine  effective  voice  data  collection  techniques  which  can  provide 
adequate  recognition  in  stressful  waving  situations  where  voice  character¬ 
istics  may  be  sign Lf icantly  altered. 

Determination  of  required  visual  simulation  field  of  view  should  also  be 
pursued  since  it  has  a  significant  influence  on  system  costs.  An  experiment, 
such  as  the  one  attempted  in  this  study  with  AWAVS,  could  be  useful  in  this 
line  of  inquiry. 

Actual  instructor  LSO  activities  while  employing  the  training  system 
should  be  analyzed  for  the  determination  of  needed  instructional  support  fea¬ 
tures  and  to  identify  the  most  effective  instructional  techniques  and  strate¬ 
gies.  This  type  of  study  should  also  provide  data  to  support  the  development 
or  refinement  of  adaptive  training  logic. 

The  system  should  also  be  employed  in  the  collection  of  skilled  LSO  and 
trainee  task  performance  data  to  support  validation  of  performance  measures 
and  refinement  of  automated  performance  evaluation  concepts.  LSO  task  per¬ 
formance  data  from  earlier  studies  should  also  be  employed  in  this  effort. 

Where  to  locate  a  prototype  LSO  training  system  is  am  important  procure¬ 
ment  question  related  to  effective  system  utilization.  Availability  of  suffi¬ 
cient  LSOs  and  trainees  for  system  evaluation  studies  is  one  factor.  Another 
factor  is  availability  of  dedicated  personnel  for  coordination  of  LS  is  and 
system  utilization,  as  well  as  to  provide  continuity  of  user  (LSO)  involvement 
with  the  studies.  Since  the  LSO  Phase  X  School  and  its  staff  have  relocated 
to  NAS  Cecil  Field,  that  location  is  recommended  for  the  system.  The  LSO 
population  for  NAS  Cecil  Field  is  reasonably  high  and  the  LSO  Train -ng  Model 
Manager  and  his  staff  can  provide  the  personnel  continuity  required  NAS 
Miramar  and  NAS  Oceana  are  also  acceptable  locations  due  to  their  h  ^gh  LSO 
population. 

The  recommended  characteristics  for  the  optimum  experimental  prototype 
LSC  training  system  are  quite  extensive  and  are  outlined  in  Appendix  C.  These 
chc  racteristics  coincide  with  the  description  of  Alternative  VI  presented  ear- 
li'  r  in  the  report.  It  is  recommended  that  consideration  be  g~ven  to  a 
twr  -stage  prototype  system  procurement.  The  first  stage  would  involve  the 
development  and  testinc  of  the  Universal  LSO  Trainer  (Alternative  V)  described 
earlier  in  this  report-  Utilization  of  this  system  could  provide  answers  to 
many  questions  concerning  operational  system  procurement  requirements.  Utili¬ 
zation  of  this  system* £  data  collection  capabilities  could  also  provide  a 
quantitative  foundatior  for  the  identification  of  effective  performance  evalu¬ 
ation  and  syllabus  control  strategies.  The  second  phase  would  involve  the 
implementation  of  automated  performance  evaluation  and  adaptive  syllabus 
control  capabilities.  The  advantages  of  this  recommendation  include  mini¬ 
mizing  costs  and  time  involved  with  acquiring  an  LSO  training  and  research 
tool  and  reduction  in  the  risks  associated  with  development  of  automated  capa¬ 
bilities  for  performance  evaluation  and  adaptive  syllabus  control. 
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APPENDIX  A 
IES  SCENARIOS 


TRAINING  SCENARIOS 

THE  FOLLOWING  SCENARIOS  ARE  DESIGNED  FOR  TRAINING  SUBJECTS  TO  MAKE 
"APPROPRIATE"  VOICE  CALLS  FOR  VARIOUS  TYPICAL  AIRCRAFT  APPROACH  DEVIATIONS 
IN  GLIDESLOPE,  LINE-UP  AND  AOA.  THE  "APPROPRIATENESS"  OF  VOICE  CALLS  TO 
VARIOUS  AIRCRAFT  STATES  IS  BASED  ON  A  SIMPLIFICATION  OF  AN  LoO  BEHAVIORAL 
MODEL.  ACTUAL  CALLS  ELICITED  BY  THESE  SCENARIOS  ARE  DEPENDENT  UPO <  TRAINEE 
ACTION  DURING  THE  APPROACH.  IN  OTHER  WORDS,  THE  ACTUAL  PROFILE  FLOWN  MAY 
DIFFER  FROM  ITS  DESIGN  DUE  TO  THE  TIMING  AND  CORRECTNESS  ASPECTS  OF  TRAINEE 
CALLS.  THE  DESCRIPTIONS  BELOW  ARE  PROFILE  DESIGNS  WHICH  MAY  ONLY  OCCUR  IF 
THERE  IS  NO  TRAINEE  INTERACTION. 

1.  SCENARIO  001LT 

HIGH  DEVIATION,  START  RANGE  ZONE  ELICITING  THE  CALL,  "YOU'RE  HIGH" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  IMMEDIATELY  STARTS  A  DEVIA¬ 
TION  TO  THE  VERY  HIGH  GLIDESLOPE  ZONE 

2 .  SCENARIO  0  0  2LT 

LOW  DEVIATION  Sr.  ART  RANGE  ZONE  ELICTING  THE  CALL  "YOU'RE  LOW” 

AIRCRAFT  STARTS  CS  GLIDESLOPE,  LINEUP,  AOA  IMMEDIATELY  STARTS  A  DEVIA¬ 
TION  TO  THE  LOW  GLIDESLOPE  ZONE 

3.  SCENARIO  003LT 

RIGHT  LINEUP  DEVIATION;  START  RANGE  ZONE,  ELICITING  THE  CALL  "YOU'RE 
LINED  UP  RIGHT" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP  AOA  IMMEDIATELY  STARTS  A  DEVIA¬ 
TION  TO  THE  RIGHT  ZONE 

4.  SCENARIO  004LT 

LEFT  LINEUP  DEVIATION,  START  RANGE  ZONE,  ELICITING  THE  CALL,  "YOU’RE 
LINED  UP  LEFT" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  IMMEDIATELY  STARTS  A  DEVIA¬ 
TION  TO  THE  LEFT  ZONE 

5.  SCENARIO  005LT 

HIGH  DEVIATION.  IN  THE  MIDDLE  RANGE  ZONE  ELICITING  THE  CALL  "YOU'RE 
HIGH" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  STARTS  A  DEVIATION  TO  THE 
VERY  HIGH  ZONE  IN  THE  MIDDLE 
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6.  SCENARIO  006LT 


LOW  DEVIATION,  IN  THE  MIDDLE  RANGE  ZONE,  ELICITING  THE  CALL  "YOU’RE 


AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  STARTS  A  DEVIATION  TO  THE  LOW 
ZONE  IN  THE  MIDDLE. 

7.  SCENARIO  007LT 

RIGHT  LINEUP  DEVIATION,  IN  THE  MIDDLE  RANGE  ZONE,  ELICITING  THE  CALL 
"YOU'RE  LINED  UP  RIGHT" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  STARTS  A  DEVIATION  TO  THE 
RIGHT  ZONE  IN  THE  MIDDLE 

6.  SCENARIO  003LT 

LEFT  DEVIATION,  IN  THE  MIDDLE  RANGE  ZONE,  ELICITING  THE  CALL  "YOU'RE 
LINED  UP  LEFT" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  STARTS  DEVIATION  TO  THE  LEFT 
ZONE  IN  THE  MIDDLE 

9.  SCENARIO  009LT 

SLOW  DEVIATION  IN  THE  MIDDLE  RANGE  ZONE,  ELICITING  THE  CALL  "YOU’RE 
SLOW" 


AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  CHANGES  TO  SLOW  AOA  IN  THE 
MIDDLE 

10.  SCENARIO  010LT 


FAST  DEVIATION,  IN  THE  MIDDLE  RANGE  ZONE,  ELICITING  THE  CALL  "YOU’RE 
FAST" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  CHANGES  TO  FAST  Ac A  IN  THE 
MIDDLE 

1 1 .  SCENARIO  011LT 

HIGH,  FAST  DEVIATION,  IN  CLOSE  RANGE  ZONE,  ELICITING  THE  CALL  "WAVLOFF" 

AIRCFAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  CHANGES  TO  FAST  AOA  AND 
STARTS  DEVIATION  TO  VERY  HIGH  ZONE  IN  CLOSE. 

12.  SCENARIO  012LT 

LOW  DEVIATION,  IN  CLOSE  RANGE  ZONE,  ELICITING  THE  CALL  "POWER" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  CHANGES  TO  SLOW  AOA  AND 
STARTS  DEVIATION  TO  THE  LOW  ZONE  IN  CLOSE- 
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1 3  SCENARIO  0 1 3LT 

LOW,  SLOW  DEVIATION,  IN  CLOSE  RANGE  ZONE,  ELICITING  THE  CALL  "WAVEOFF" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  CHANGES  TO  SLCW  AOA  AND 
STARTS  DEVIATION  TO  THE  LOW  ZONE  IN  CLOSE 

14.  SCENARIO  014LT 

LEFT  LINEUP  DEVIATION,  IN  CLOSE  RANGE  ZONE,  ELICITING  THE  CALL  "RIGHT 
FOR  LINEUP" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  STARTS  DEVIATION  TO  THE  LEFT 
ZONE  IN  CLOSE 

15-  SCENARIO  01SLT 

RIGHT  LINEUP  DEVIATION,  IN  CLOSE  RANGE  ZONE,  EL 'CITING  THE  CALL  "LEFT 
FOR  LINEUP" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  STARTS  DEVIATION  TO  RIGHT 
ZONE  IN  CLOSE 

16.  SCENARIO  016LT 

HIGH,  RIGHT  DEVIATION,  IN  CLOSE  RANGE  ZONE,  ELICITING  THE  CALL  "WAVEOF1’" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINE-UP,  AOA:  STARTS  DEVIATION  TO  THE 
VERY  HIGH  AND  RIGHT  ZONES  IN  CLOSE 

17.  SCENARIO  0 1 7LT 

LOW,  LEFT  DEVIATION,  IN  CLOSE  RANGE  ZONE,  ELICITING  THE  CALL  "WAVEOFF” 

AIRCRAFT  STARTS  ON  GLIDESLOPE.  LINEUP,  AOA  STARTS  DEVIATION  TO  THE  LOW 
AND  LEFT  ZONES  IN  CLOSE. 

18.  SCENARIO  018LT 

HIGH  DEVIATION,  AT  RAMP  RANGE  ZONE,  ELICITING  THE  CALL  "WAVEOFF" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  STARTS  DEVIATION  TO  VERY  HIGH 
ZONE  AT  THE  RAMP. 

19.  SCENARIO  019LT 

LOW,  DEVIATION,  AT  RAMP  RANGE  ZONE,  ELICITING  THE  CALL  "WAVEOFF" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  STARTS  DEVIATION  TO  THE  LOW 
ZONE  AT  THE  RAMP 
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JO  SCENARIO  020LT 

RIGHT  DEVIATION,  AT  RAMP  RANGE  ZONE,  ELICITING  THE  CALL  "WAVEOFF" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  STARTS  DEVIATION  TO  RIGHT 
ZONE  AT  THE  RAMP. 


21.  S'ENARIO  G21LT 

LEFT  DEVIATION,  AT  RAMP  RANGE  ZONE,  ELICITING  THE  CALL  "WAVEOFF" 

AIRCRAFT  STARTS  ON  GLIDESLOPE,  LINEUP,  AOA  STARTS  DEVIATION  TO  THE  LEFT 
ZONE  AT  THE  RAMP 

22.  SCENARIO  022LT 

SLIGHT  DEVIATIONS;  LINEUP  AND  GLIDESLOPE  START,  MIDDLE 
NO  CALLS  EXPECTED 

23.  SCENARIO  023LT 

SLIGHT  DEVIATIONS  GLIDESLOPE  AND  LINEUP;  START,  MIDDLE 
NO  CALLS  EXPECTED 

24.  SCENARIO  024LT 

SLIGHT  DEVIATIONS;  LINEUP  AND  GLIDESLOPE;  START,  MIDDLE,  CLOSE 
NO  CALLS  EXPECTED 

25.  SCENARIO  02  5LT 

SLIGHT  DEVIATIONS,  GLIDESLOPE  AND  LINEUP;  START,  MIDDLE,  CLOSE 
NO  CALLS  EXPECTED. 

26-  SCENARIO  02o*,T 

SLIGHT  DEVIATIONS,  LINEUP  AND  GLIDESLOPE;  START,  MIDDLE,  CLOSE 
NO  CALLS  EXPECTED 

.  SCENARIO  0 2 7LT 

SLIGHT  DEVIATIONS  GLIDESLOPE  AND  LINEUP;  START,  CLOSE 
NO  CALLS  EXPECTED 
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2  < .  SCENARIO  028LT 

SLIGHT  DEVIATION;  GLIDESLOP  :  AND  LINEUP,  START,  MIDDLE,  CLOSE 
NO  CALLS  EXPECT CD 

29.  SCENARIO  029LT 

"OKAY"  PASS  (ON  GLI DESLOPE,  LINEUP,  AOA  THROUGHOUT) 

MEDIUM  PILOT  DEVIATIONS  WITHIN  THE  "ON"  ZONES 
NO  CALLS  EXPECTED 

30.  iCENAR  C  C30LT 

"OKAY"  PASS  (ON  GL ( DESLOPE,  LINEUP,  AOA  THROUGHOUT) 

VARIED  PILOT  DEVIATIONS  WITHIN  THE  "ON"  ZONES 
NO  CALLS  EXPE  ITED 
TESTING  SCENARIOS 

THE  HOLLOWING  SCENARIOS  .RE  DESIGNED  FOR  TESTING  THE  GENERALIZ ABILITY  OF 
SIMPLE  "WAVING"  SKILLS  ACQUIRED  WITH  LAB  TRAINING  SCENARIOS  TO  MORE  COMPLEX 
"WAVING"  SITUATIONS.  THE  SCENARIOS  ARE  BASED  ON  "TYPICAL"  ERROR  TREND 
PROFILES,  SEVERAL  OF  WHICH  CAN  LEAD  TO  RAMP  STRIKES  AND  BOLTERS  IN  ACTUAL 
OPERATIONS. 


ACT'  AL  CALLS  ELICITED  BY  THESE  SCENARIOS  ARE  DEPENDENT  UPON  TRAINEE  ACTION 
DURING  THE  APPROACH.  IN  OTHER  WORDS,  THE  ACTUAL  PROFILE  FLOWN  MAY  DIFFER 
FROM  ITS  DESIGN  DUE  TO  IHE  T ICING  AND  CORRECTNESS  ASPECTS  OF  TRAINEE  CAULS. 
THE  DESCRIPTIONS  BELOW  ARE  PR  'FILE  DESIGNS  WHICH  MAY  ONLY  OCCUR  IF  THERE  IS 
NO  TRAINEE  INTERACTION 

1.  SCENARIO  30’PT 

LINED  A  LIT'  RLrHT  AT  START 
LINED  "■'?  RIGhT  IN  MIDDLE,  IN  CLOSE 
LINED  C?  LEFT  T  RAM? 

2.  SCENARIO  C3  2PT 

CLOW  IN  MIDDLE,  LOW  IN  MIDDLE 
LOW,  FAST  IN  CL  SE,  AT  RAMP 

3.  SCENARIO  C  0 3 PT 

LITTLr  HIGH  START 
HIGH  IN  M I  OGLE 
LOW  AT  RAMP 
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SCENARIO  004PT 

LITTLE  HIGH  IN  MIDDLE,  IN  CLOSE 
LOW  AT  RAMP 

SCENARIO  005PT 

LITTLE  LOW,  LINED  UP  RIGHT  AT  START,  IN  MIDDLE 
LOW  IN  CLOSE  ON  LINEUP  CORRECTION 
LOW  AT  RAMP 

SCENARIO  006PT 

LITTLE  LOW  AT  START,  IN  MIDDLE 
HIGH,  FAST  IN  CLOSE,  AT  RAMP 

SCENARIO  007PT 

LITTLE  LINED  UP  LEFT  IN  MIDDLE 
LINED  UP  RIGHT  IN  CLOSE,  AT  RAMP 

SCENARIO  008PT 

LOW  AT  START,  IN  MIDDLE 
LOW,  SLOW  IN  CLOSE,  AT  RAMP 

SCENARIO  009PT 

LITTLE  HIGH  IN  MIDDLE 

SLOW  IN  MIDDLE 

LINED  UP  RIGHT  IN  CLOSE 

LOW,  LINED  UP  RIGHT  IN  CLOSE,  AT  RAMP 

SCENARIO  01  OPT 

LINED  UP  LEFT  AT  START,  IN  MIDDLE 

SLOW  IN  MIDDLE 

LOW  IN  CLOSE,  AT  RAMP 
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10.  After  going  through  this  experimental  LSC  training  program,  do  you  feel 
that  you  actually  have  a  better  conceptual  understanding  of  the  LSO’s 

"waving”  job?  _  If  yes,  do  you  feel  that  using  the  experimental 

system  was  tne  major  factor  in  your  learning  experience  (as  opposed  to 
picking  up  "waving"  concepts  through  briefings  and  discussions  this 
week)?  _ 


11.  Do  you  think  this  system  would  be  an  effective  part  of  introductory  LSO 
decision  training?  Why /why  not? 


12.  What  suggestions  do  you  have  for  improving  the  conduct  of  this 
experiment? 


13.  Any  other  final  comments? 
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vraft  to  obtain  adequate  LSO  performance.  The  last  issue  to  be  addressed 
vc-r. terns  fee  effect  of  field  of  view  (FOV)  information  on  LSO  performance  ... 
/  .suai  simulation. 

■  ;etn  ed  -  Experiment _ 1_.  Design :  This  experiment  will  involve  a  five  var .  abi 

randomized  block  factorial  design.  The  first  variable  to  be  considered  is 
so  ;t ion .  That  is,  resolution  measured  in  terms  of  the  number  of  raster 
•■its  utilised  in  projecting  the  target  (A-7 )  image.  The  target  projector  : 
•WAVS  will  be  operated  in  its  zoom  mode  (i.e.  ,  not  a  fix«d  area  projects  on) . 

-com  variable  is  the  simulated  ambient  light  condition  (dav/ragbt)  of 
-'£  .■■■■'■■  r  art  .-cenario.  Two  levels  of  this  variable  will  be  consider  ad,  .  nr. 

.  f hi  -s r.  ;■■•■■ ,-.  a  .lay  seem-  and  the  other  representing  a  nigr.t  scene.  01  star. .•« 
-cm  touchdown  when  a  significant  deviation  occurs  is  the  third  variable 
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Procedure:  Prom  .  ‘  t,S0  ’  s  point  v;  ew  •  he  <  x  •t-r  .  «  r,  •  a  .  •a.-.x  .  . 

resemble  the  wdVir,r;  td:>k  he  pcr'OMiic  while  ■  or.tr'..  ■  r% .  •  - 

craft  land  iii;  s.  The  LBf  will  observe  the  .1  r-.aft  i  ■  .  t  •  ,  a:.;  .  ,i. 

cake  cal :  s  that  indicate  a  pert*  ption  c.l  a  lev .an*  ,1 :  >  •  -a :  .et«  •  -r  :  • 

this  situation  the  aircraft  will  begin  dt-vi  it.  n*.  •  r  - ..  t  .  leu.  .  .  ■ 
cont  i  role  to  deviate  un:.:  .  .  1;  to:  LSO  mak  a  •  a  .  i  •  .  ,  •  .  •••: 

a  pre- defined  zone  boundary.  Wrier.  eitner  o:  th«  ,<• 

display  wall  clank  for  ’:ivc  st  -onds .  t’o  i  1  ow . .  «.  .  . .  • 

res  :,r.e  the  approach  frot  tr.e  zone  Dour.dary  a.-vi  •  .  .e  .  .  .  • 

.10  .  urt w  c  c  Uiiv  la "  iop.5'  *"  '“orr.  cr.»>  'j  *  *  it i> 

delay  after  an  approach  has  jh«p.  completed  act  ore  *  t  if. a1  :  u  :  a 

Thus  the  LSO 1  s  task  is  to  observe  the  d : spl  a'  ar.o  a:-  :  a’ 

The  subject's  performance  will  be  assessed  «.>  r  ,  r  :t.  r  ■.  »•  -u-.i  ■'  t 

cal*  and  when  the  call  is  made.  In  addif.cn,  .  1.  •*•.-.  r.  ....  .  .  ,• 

as  to  the  reasons  for  his  calls,  r.is  subjecr  ive  .  ayress  *  .r  s  .  •  t  •  «  • 

etc .  We  would  also  like  to  monitor  the  sub  ,-ct  *  s  eve  movvw.r  s  . 

ment  of  wuer.  the  call  is  made  should  irecm  w;  e.i  .  a 1  e  ;  -  a".  : 

at  the  onset  of  the  response.  The  response  m.-asu'  .  ,  -ieul  .  tv  t*.  t.n  ■  :r .  . 
second  level  to  allow  for  an  accurate  conpar , s  n  Lriwcan  croups  a.-.l  jaui.r  . 
assessment  of  the  appropriate  aircraft  parameters.  T.>e  ust.  or  a  v  lice.  a  •  - 
vated  switch  will  facilitate  the  measurement  of  the  timing  and  r«i-or'..i.c 
the  response.  When  an  approach  is  completed  the  LSI  will  be  asked  -a  few  per¬ 
tinent  questions  concerning  the  approach.  Ten  seconds  later  the  next  apy  i r. 
begins  and  the  process  is  repeated  The  iteration  continues  for  twenty  ap¬ 
proaches.  Of  these  twenty  approaches,  ten  feature  ar.  aircraft  representa*  «or, 
from  the  first  level  of  detail  and  ten  from  the  second  level.  The  order  f 
presentation  of  these  various  aircraft  representations  is  to  oe  ranoom  w,th.,u\ 
rep! acement . 

Within  these  twenty  trials,  the  call  range  variable  is  manipulated  with  four 
profiles  representing  each  zone.  The  remaining  four  approaches  will  be  v.-.t;.-. 
trials  where  the  deviations  that  occur  are  slight  and  are  not  expected  to 
elicit  a  call  from  the  LSO.  The  order  of  presentation  of  these  trials  is  t 
be  random,  again  without  replacement.  Of  the  four  trials  for  each  range  cone, 
‘wo  are  to  be  with  the  finely  detailed  aircraft  representation  and  two  with 
the  "wire  aircraft"  representation.  The  same  type  of  arrangement  .  s  expected 
for  the  catch  ..rials. 


•\  group  of  twenty  triads  constitutes  a  session  ana  each  session  is  iui  under 
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resolution  level,  one  half  of  the  subjects  are  to  ream  the  oast  y-Scytivs  at  a 
high  level  ar.c  the  other  half  at  a  low  level.  Thereafter,  the  '.  vel  .  to  be 
ma:,:.pu*at  id  across  sessions  in  an  ABBA  manner  for  each  subject. 

T.".o  day/r.icfiit  variable  must  be-  balanced  wit.l._n  subjects  and  a  irons  b. 'tks  of 
tr.ils-  Bach  subject  is  given  four  blocks  of  trials.  Each  block  of  tiua'i? 
will  be  run  under  either  day  or  night  conditions.  Half  the  subjects  will  re¬ 
ceive  day  conditions  in  the  first  block  and  r.alf  will  receive  night 


59 


NAVTRAEQUIPCEN  78-C-0151-1 


"SK 


conditions.  Thereafter,  the  order  of  day/night  presentations  will  oe  governed 
by  the  ABBA  balancing  rule.  Table  C-1  presents  a  summary  of  the  experimental 
procedure . 

Subjects:  Six  randomly  selected  LSOs  will  serve  as  the  subjects  for  this 

?xperiment.  The  LSOs  should  have  attained  at  least  a  Wing  LSO  designation 
level  and  should  be  proficient  (performed  LSO  duties  within  past  year) . 

Apparatus:  AWAVS ,  visual  simulation  research  facility  at  NAVTRAEQUIPCEN. 

stimuli:  An  aircraft  representation  generated  by  the  CGI  portion  of  AWAVS 

will  be  usea  as  the  primary  stimuli  for  the  experiment.  Two  aircraft  images 
will  be  used  as  discussed  above.  The  set  of  approach  profiles  will  be  de¬ 
signed  by  Logicon.  The  background  field  of  view  is  to  be  set  at  its  maximal 
letting  and  is  to  include  the  carrier  deck  outline,  horizon,  ship's  wake, 
arresting  wires  and  deck  markings.  The  optimum  aircraft  touchdown  point 
shoui i  be  within  the  viewing  area. 

Method  -  Experiment  2.  Design:  Experiment  2  utilizes  a  fcur  variable,  ran¬ 
domized  block  design  similar  to  tne  design  of  Experiment  1.  Three  of  the 
variables  are  the  same  as  those  found  in  the  first  experiment:  day/night, 
range  zone  and  subjects.  In  this  experiment,  field  of  view  is  substituted  for 
resolution  and  level  of  detail.  The  field  of  view  (FOV)  variable  is  a  fixed 
variable  with  three  levels  of  consideration,  each  corresponding  to  a  specific 
informational  component.  The  first  level  of  this  variable  contains  the  maxi¬ 
mum  amount  of  referent  information  including  the  deck  wires,  the  touchdown 
zone,  the  deck  with  the  lineup  stripe,  the  deck  outline,  and  the  ship's  wake. 
Condition  two  includes  all  of  the  above  except  the  wire  and  the  touchdown 
zone.  This  condition  contains  information  to  help  the  LSO  with  lineup  (the 
centerline,  the  deck  outline  and  the  wake),  but  it  does  not  provide  informa¬ 
tion  concerning  the  touchdown  referent  or  the  feedback  from  the  aircraft 
catching  the  wires.  The  last  level  of  this  variable  is  the  most  information¬ 
ally  impoverished  condition  in  that  only  the  port  aft  corner  of  the  ramp  and 
the  ship's  wake  is  available  to  the  LSO.  Level  three  corresponds  approxi¬ 
mately  to  the  FOV  available  in  a  single  CRT  visual  system  and  level  two  ap¬ 
proximates  the  FOV  of  a  two  CRT  visual  system.  In  all  conditions  a  portion  of 
the  FOV  is  allocated  to  the  right  of  the  LSOs  line  of  sight  to  include  the 
ship's  wake  and  urea  of  likely  aircraft  deviation.  The  experime  t  will  be  run 
using  the  higne resolution  AWAVS  can  muster,  and  the  target  will  oe  depicted 
with  the  most  cetail  possible.  Figure  C- 2  graphically  presents  the  design. 

Procedure:  The  procedure  in  this  experiment  is  similar  to  the  procedures  in 

Experiment  1  m  that  we  are  again  dividing  up  the  experiment,  into  blocks, 
sessions  and  trials  (approaches).  This  time  each  session  will  include  live 

approach  profiles.  These  five  approaches  include  one  with  a  deviation  m  each 
of  the  four  range  zones  and  a  catch  trial  with  no  significant  deviations.  The 
five  trials  that  constitute  a  session  will  be  run  ur.de  a  single  .  TV  1  vel. 
There  will  be  ten  sessions  in  a  single  block ,  one  warmup  session  and  mr.e  test 
sessions.  Each  Dloc-k  is  run  as  a  day  block  or  a  night  Mock .  The  experiment 
consists  of  four  Mocks,  two  night  blocks  and  two  day  fclocxs. 

Given  that  we  have  two  block;,  each  for  day  or  night  conditions,  and  that  each 
block  contains  nine  test  sessions,  we  can  completely  balance  the  order  of  FOV 
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TABLE  C-1.  rLOCED’JKL  LUMMAiW  1  Or  EXPERIMENT  '. 


A;  ixpcr  icvr  t  I  -  4  blocks  of  f  r ;  a  I  s/per  subject 


r  b  i  o  c  a  is  i  r.\  under  a  ,  ^ ■  r  a  -Lv  k  j  _  light  cor.  Li.:  vn. 


iunifects  i  n  the  experiment 

-  Within  suc.iects  day  versus  night  Conditions  i s  balanced  across 
blocks  i.  AX'  a  A  )  . 

-  Across  subiects  the  day/night  condi  cion  cf  the  first  block  is 
balanced . 


i  block  -  i  sessions 

-  "i  warmup  session  and  4  test  sessions. 

-  Level  of  resolution  is  balanced  across  sessions  (ABBA) . 

-  Level  of  resolution  of  the  first  session  is  balanced  across 

s ut> ;  ec  t  s  • 


O'  1  session  -  20  approaches  (100  approaches  per  block) 

-  100  approach  profiles  will  be  generated  with  20  profiles  being 
randomly  assigned  to  each  session. 
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levels  (3!)  across  the  two  blocks.  Naturally,  the  sequence  of  condition 
orders  must  also  be  balanced  across  subjects.  For  example,  if  we  assign  the 
FCV  level  order  1,  2,  3  to  the  name  A  and 

2,  3,  1  -  B 

3,  1,  2  -  C 

1  ,  3,  2  -  D 

3,2,  1  -  iS 

2,  1,  3  -  F 

then,  we  can  proceed  to  balance  the  presentation  of  these  orders  across  sub¬ 
jects  using  a  latin  square  procedure. 

Subjects;  Twelve  subjects  are  required.  Bach  subject  must  meet  the  same 
standards  as  described  m  Experiment  1. 

Apparatus:  Same  as  Experiment  1. 

ACTUAL  EXPERIMENT  AND  RESULTS 

Unfortunately,  these  experiments  were  not  conducted  as  originally 
planned.  The  AWAVS  device  was  not  capable  of  handling  many  of  the  variaDles. 
Resolution  turned  out  to  be  a  meaningless  dimension  in  the  AWAVS  environment 
since  *-he  best  resolution  possible  was  barely  adequate  for  the  task.  The  next 
best  level  of  resolution  was  totally  inadequate  in  an  LSO  context.  The  day- 
vs-night  variable  was  also  dropped  since  the  simulation  was  not  equipped  to 
show  a  night  scene  or  a  night  aircraft  image.  Level  of  detail  variation  was 
also  beyond  the  simulation's  capability  since  only  one  aircraft  image  was 
available.  The  distance  from  touchdown  variable  was  discarded  since  the  tar- 
-at  projector  had  to  remain  fixed  in  a  single  position.  In  addition,  it  was 
discovered  that  the  software  which  controlled  the  target  aircraft  only  allowed 
a  straight-vector  line  approach.  That  is  to  say,  once  the  aircraft  started  on 
an  approach  vector  it  would  not  accept  a  modification  of  that  vector  during 
the  approach.  There  was  no  method  of  randomly  selecting  approach  profiles  m 
real  time.  The  sequence  of  approaches  had  to  be  recorded  prior  to  the  start 
of  the  experiment.  All  of  these  constraints  led  to  a  single  experiment  which 
is  described  below. 

design.  The  -jor  independent  variables  were: 

1  •  Fiei-..  of  View  -  The  field  of  view  was  as  described  in  Experiment  2 
riiioie,  ana  aeccnp- isr.ed  oy  means  of  taping  a  sheet  of  cardboard  over  the 
ar-prot r Late  portion  of  the  background  projector  lens.  Three  levels  of  back¬ 
ground  ~  ie*. a  view  were  used.  Each  of  these  levels  correspond  to  ct-ecific 

v  ■  of  info  .-mat  ion  that  the  LSO  may  u«-»  to  calibrate  h-s  perceptuc.'.- 
.  .•  :n- .  -.I--  In  .Moc.v-i.t :  The  first  level  provider.  r complete"  in  fc. nation  by 

:  .lading  *  .'.cw  from  the  duck  wires  to  about  20  degrees  right  ;f  ap- 

:  -r.  .ng  .  .-aft  ,  u  viewing  are-  of  160  degrees-  Thu  sectn c.  ,e. it.uced 
..  . :  ‘  or r. . : : by  cutti  ng  out  the  landing  area  and  the  ■ent  ,  .  ii  the 

-  .incut  90  degrees .  The  third  level  reduced  f.nc  information 

further  -o  about  a  45  degree  field  of  view  in  the  approach  area. 
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2.  Type  of  Deviation  -  Since  only  single  vector  approaches  were  possi¬ 
ble,  there  were  two  types  of  deviations  introduced.  Lineup  and  Glideslope. 

3.  Size  of  the  Deviation  When  the  Approach  Was  Terminated  -  Since  the 
approach  could  not  be  completed  to  touchdown,  all  approaches  were  terminated 
on  LSC  call  or  at  a  set  distance  (approximately  1500  :eeti  from  touchdown  f 
no  call  was  maae.  The  size  of  the  deviation  at  that  ,,omt  was  determined  r  rom 
positioning  coordinates.  There  were  four  levels  of  this  variable:  large, 
medium  and  small,  as  well  as,  no  deviation  (a  catch  trial).  Basis  for  devia¬ 
tion  size  was  based  on  questionnaire  data  collected  in  an  earlier  study  by 
Hooks  and  otners  (1978). 

The  dependent  variables  were: 

1.  The  LSO  Calls  -  The  subjects  were  instructed  to  make  calls  to  the 
aircraft,  just  as  though  they  were  actually  waving  the  approach.  No  knowledge 
of  results  (touchdown  information)  was  provided.  The  aircraft  appeared, 

"flew"  the  profile  and  disappeared  at  the  set  range  or  on  the  voice  call. 

2.  The  LSO's  Recall  of  the  Deviations  During  the  Approach  -  After  each 
approach  the  subject  was  asked  for  a  profile  description,  noting  any  portions 
of  the  approach  where  he  felt  the  aircraft  may  have  deviated  from  the  ideal 
glideslope  or  lineup. 

Stimuli:  Ar,  aircraft  representation  generated  by  the  CGI  portion  of  AWAVS  was 

used  as  the  primary  stimuli  for  the  experiment.  The  AWAVS  target  image  pro¬ 
jection  field  of  view  was  set  at  its  maximal  setting.  The  background  field  of 
view  included  the  carrier  deck  outline,  horizon,  ship's  wake,  arresting  wires, 
and  deck  markings.  Thirty  approach  profiles  were  constructed  for  use  in  this 
experiment.  The  profiles  were  computer  generated  images  of  an  A-7  flying  an 
approach  to  a  Forrestal-class  aircraft  carrier.  The  approaches  differed  from 
one  another  in  two  ways,  the  range  at  which  the  approach  began  and  the  type  of 
deviation  that  occurred  during  the  approach. 

The  start  range  and  deviation  types  were  designed  to  combine  in  a  completely 
balanced  manner.  That  is,  each  type  of  deviation  began  at  each  of  the  differ¬ 
ent  start  ranges.  For  example,  the  "high"  glideslope  deviation  approach  Degar. 
at  each  of  the  three  .tart  ranges,  resulting  in  three  "high"  gl  -  devia¬ 

tion  profiles.  All  profiles  were  terminated  at  1500  feet  from  tr.e  tc.jcr,dowr. 
point-  Therefore  the  tnree  "high"  glideslope  profiles  each  reflected  differ¬ 
ing  amounts  of  deviat.on  at  this  termination  point.  Catch  trial  approaches 
were  also  included.  Che  start  ranges  for  these  approaches  were  keyed  to  those 
for  approaches  which  had  deviations.  Within  the  thirty  profiles,  six  r.ad  only 
glideslope  deviations,  six  had  only  lineup  deviations,  and  nine  were  combina¬ 
tions  of  glideslope  and  lineup  deviations.  There  were  nine  catch  triads. 

Subject:  Six  experimentally  naive  LSOs  spent  two  days  in  Orianio,  Flor.da,  to 

participate  m  the  experiment. 

Procedure:  Each  of  the  six  subjects  was  run  under  all  conditions  of  back¬ 

ground  field  of  view.  The  subject's  task  in  this  experiment  was  twofold. 
First,  he  observed  the  approach  and  made  a  call  when  the  aircraft  deviated  far 
enough  to  require  correction  or  warning.  When  the  call  was  made,  the  approach 
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Figure  C-3.  Probability  of  Making  a  Call 
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APPENDIX  D 

EXPERIMENTAL  PROTOTYPE  LSC  TRAINING  SYSTEM 

. i s  .»ppr. r.dix  describes  the  performance  capabilities,  recommended  for  an 
••xper l-.’.er.cal  prototype  LSO  training  system.  The  capabilities  are  organized  in 
CT:  same  functional  structure  reported  by  Hooks  and  others  (’978).  Figure 
L- * .  excerpted  from  that  report,  is  provided  as  an  organizational  aid  for  the 
review  of  information  presented  below. 

IN  JTA JCT ICnAL  '.'•RESEnTA.TiCN 


The  instructional  presentation  aspect  of  the  system  involves  presentation 
.■r  cu-..- s  which  enable  trainee  task  performance  and  promote  the  learning  of  LSG 
&.<ills  and  knowledges.  The  functional  elements  are  described  ieiow. 


a  1  Environment.  Field  of  view  should  encompass  the  continuum  from 
approach  corridor  through  the  touchdown  zone. 


Approaching  Aircraft  (under  LSC  control) 


aircraft  types:  A-6 ,  A-7 ,  E-2,  F-14,  S-2  these  are  the  primary 
fleet  aircraft;  others  which  have  been  left  out  are  not  con¬ 
sidered  necessary  for  experimentation). 


(2)  position  lights;  red,  green  and  white  lights  located  on  dif¬ 
ferent  parts  of  the  aircraft;  positioning  of  the  lights  varies 
by  aircraft  type;  intensity  and  operability  of  individual  lights 
under  instructional  control  of  the  system. 


3)  AGA  lights:  rea,  amber  and  green  lights  located  on  front  por¬ 
tion  of  aircraft  (usually  the  nose  wheel  strut  area),  intensity 
of  the  ) lgrfcs  and  operability  of  individual  lights  under  in¬ 
structional  control  of  the  system;  lights  correlated  to  aircraft 

h  OC  t:  d  . 


( 4 ;  iircraft  dynamics:  pitch,  roll,  yaw  angles;  speed. 

Cr.er  .  t-n..  Environment. 

( '•  :  c.r.1. :  cot  light:  night  conditions. 

(2.  vcat  .t.r  effects:  reduced  visibility,  ceiling. 

■  j)  .iori.no.’,  definition:  continuum  from  well-oefined  to  r.o:.- 
t;xi  stent. 

;  **  >  ; --are  au,Hu  ciestroyer-.  snip  positioned  apprex isr.at  1  y  «. .  -  tiJr 

him r.rf  carrier;  red  mast  lights. 

Ai  rcr 'i*- 1'.  C. i. *  tr 


out.  i.-.o .  deck  edge  in  LEO  field  of  view:  only  one  carrier 
:•  rent:  to  be  modelled. 


n  1 


I  1  -  1 


c  iUc i o .id  i.  £ f  r c .  ■  'j 

u  1  pha r. une ncs;  ror  cu» t a  at.'ii.  «.* c  J;. .  w.  •*<.;/  i n s t i  «. 

t  1 0  f:  d  1  O  iitr  0 1  ♦ 

Cj  rctors  ■;  •. '  *.o:.s  .  .  r  cr..i  r  :  c  '  e  a 

pe  r-‘ v*v;  ual  ‘-e  c  ncj  . j  *  -j'j*  *  nulr.  j 

0"f  idu  ’■  positioning;  d^$piay  .ii.vOrx' 

system. 

ft .  ' .  e  r.  1  i-  o  1  or  .  •  i  I  t  t a  :  r  c  r  a  t  *  }  y :  a.:  i  r  .$  dnj  Lb  J  /  pilot  l  r.  tera; 

at  i  able.  (from  t  wo  independent  Soviets  r  ir.structor-oxnt.r  ~  1  led  1 
ompute r  s  ir.u lat  ion . 

cratt  Performance  rharacr.er :.st. . 

aircrart  typos.  as  indicated  in  Visual  Environment 

.  .  'ij . t  ' * a  r a c  ter  if-t  ics  ;  p i  tc ■: . ,  i c » 1  .  yaw  s pe e cl 


vetoes 


i .  v  r.  ■;  nro  :  ci  n  n  a  r .  I  -  r .  <.  ..  o  e  .  c;  o 
-  x.  i'cquii  Gci;  can  be  i  us  true  t.oi 
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h.  Environmental  sounds 

(1)  engine  of  approaching  aircraft:  variations  in  pitch  and  inten¬ 
sity  r-orrelated  to  range  and  pilot  throttle  control; 
interactive  with  approach  situation. 

(2  deck  noises:  aircraft  and  flight  deck  vehicles;  non¬ 
interactive. 

c.  Instructor/student  communications 
Workstation  Displays. 

a.  WOD  Indicator:  wind  speed  and  direction  relative  to  ship  heading. 

b.  Hook-to-Ramp  Indicator:  dynamically  shows  relative  positioning  of 
ramp  to  optimum  glideslope. 

c.  FLOLS  Indicators:  basic  angle  and  roll  angle 

d.  SPN-42  Radar  Indicators:  speed  (true  or  closure),  line-up  devia¬ 
tions,  glideslope  deviations. 

e.  Waveoff  Indicator:  red  light  near  LSO  console  indicative  of  waveoff 
light  activation. 

f.  MOVLAS  Position  Indicator:  indicative  of  MOVLAS  signal  positioning. 
Workstation  Controls. 


a-  "Pickle":  hand-held  device  for  activating  waveoff  and  cut  lights. 

b.  MOVLAS  Control:  hand-operated  lever  for  signalling  perceived  (LSO) 
glideslope  position  of  aircraft  and  inducing  pilot  responses. 

c.  Radio:  hand-held  voice  transitu. t-receive  device  for  radio 
communications . 

Deck  Motion  Cues. 


a.  Roll:  dynamic  rotation  and  static  positioning  (trim)  about  longitu¬ 
dinal  axis  of  ship. 

b.  Pitch:  dynamic  rotation  and  static  positioning  (trim)  about  lateral 
axis  of  ship. 

Instructional  Feedback - 

.  5  /ai native  Feedback  scoring,  diagnosis  and  specific  performance 

information,  situation  recorc./replay ;  freeze. 

b.  Demonstration:  presentation  of  ideal  performance  and  typical  per- 
formance  errors,  through  "canned"  approaches. 
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APriVE  CONTKOL 

Adapt  ■.  vi-  control  involves  the  selection  and  control  of  conditions  and 
rustr  ns  tailored  to  the  instructional  needs  of  the  trainee.  Both  automated 
d  manual  elements  are  described  below. 

li-ctior.  of  learning  Alternatives. 

a  Tasx  Selection:  selection  of  the  next  skill  to  he  learned  or  prac- 
bused  on  prior  performance. 

h.  I. . struct lO.ial  Strategy  Selection:  selection  of  the  exercises  and/or 
structior.al  effects  which  promote  learning  cf  the  skill  selected. 

..  t  motor  .  nterventior. .  Functions  enabling  the  instructor  to  act  as  an 
aptive  training  controller. 

a.  Display:  information  concerning  instructional  strategy,  exercise 
editions,  trainee  performance;  to  include  repeater  of  student  view  of 
proa  cf. . 

b.  Control ;  manual  control  of  information  accessibility  and  exercise 
'.c.-.  i-ons  ;  visual,  pilot/aircraft,  audio,  trainee  workstation  displays/ 
ntrois,  deck  motion  and  instructional  effects  as  descri oed  earlier). 

kfORMANCE  EVALUATION 

Functional  support  for  evaluation  of  perceptual,  decision-making  and 
spo.ise  skills. 

:  f  rrma nee  _M eaiureir.ent . 

a.  vaic-e  Calls:  extraction  of  speech  recognition  data  relevant  to  LSO 
die  voice  calls. 

b.  Control  Activations :  extraction  of  data  relevant  to  LSD  control 
ti  vations  CY'LkS,  waveoff  lights,  cut  lights,  radio). 


l.  extraction  of  aircraft  positioning  ar.u  it  ate 


formation . 


d.  Instructor  Grading:  manual  instructor- generated  grading  inputs. 


■  finance  Predict  loos :  specif .cation  of  predicted  LUG  cer.orciance 


:IZ  1.  v '  '  ■  ca:e 


.se  conditions. 


itonsdice  Comparison:  comparison  of  performance  measurement  aata 
i.  grading  to  predicted  performance. 
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